diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index ca00ca7b..695ac825 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -42,12 +42,18 @@ jobs: - name: zig fmt --check run: zig fmt --check src/main.zig src/mcp.zig build.zig + - name: Zig source size guard + run: bash scripts/check-zig-lines.sh + - name: zig build test run: zig build test --summary all - name: JSON live-control protocol test run: python3 scripts/test-json-controls.py zig-out/bin/graff + - name: Live JSON stream contains no raw stdout lines + run: python3 scripts/test-json-live.py zig-out/bin/graff + - name: PTY spinner anti-stealth test run: python3 scripts/test-pty-spinner.py zig-out/bin/graff @@ -60,6 +66,12 @@ jobs: - name: PTY model preference and fallback test run: python3 scripts/test-model-preference.py zig-out/bin/graff + - name: Codex WS/SSE transport and mid-turn compaction test + run: python3 scripts/test-pty-codex-ws.py zig-out/bin/graff + + - name: AI title and first turn run in parallel + run: python3 scripts/test-title-parallel.py zig-out/bin/graff + - name: Stream stall/drop is not a user Esc (#134/#132) run: python3 scripts/test-stall-drop.py zig-out/bin/graff diff --git a/.gitignore b/.gitignore index ad81a533..722dae9b 100644 --- a/.gitignore +++ b/.gitignore @@ -27,6 +27,7 @@ __pycache__/ .wrangler/ # local agent/IDE config (not project source) +.codex/ .devin/ # release artifacts diff --git a/Agentic_Operating_System.md b/Agentic_Operating_System.md new file mode 100644 index 00000000..70d6d591 --- /dev/null +++ b/Agentic_Operating_System.md @@ -0,0 +1,452 @@ +# The Agentic Operating System +## From Generative AI to Autonomous Orchestration + +**Status:** Working architectural draft +**Scope:** A general framework for agent runtimes, grounded in Codegraff's current architecture +**Core claim:** The next useful abstraction is not “a chatbot with tools.” It is an operating system for agentic execution. + +--- + +## Abstract + +Large language models are powerful reasoning engines, but an LLM by itself is not an agent. A model call is stateless: it receives bytes, predicts bytes, and returns. Agency emerges only when that model is embedded inside a runtime that can preserve state, execute tools, observe results, recover from failure, manage memory, spawn workers, and improve its own behavior over time. + +This document calls that runtime an **Agentic Operating System** (**AOS**). + +An AOS is the layer that turns a passive model into an active computational process. It is responsible for converting human intent into coordinated action. In the same way that a traditional operating system turns CPU instructions into files, processes, devices, sockets, permissions, and persistence, an Agentic Operating System turns model intelligence into tasks, tool calls, subagents, traces, memory, evaluations, and self-improving behavioral policies. + +Codegraff is an early concrete example of this pattern. Its core agent loop, subagent/workflow system, tool and MCP layer, context compaction, trace files, trajectory archive, and Darwin-Gödel-style prompt evolution together form the skeleton of an AOS. + +--- + +## 1. Why the chatbot abstraction is too small + +The default interface for LLMs is still the chatbot: a user writes a message, the model writes a response. This is useful, but it hides the more important architectural question: **what is responsible for turning a response into progress?** + +A chatbot can suggest a command. An agentic runtime can run it, inspect the output, decide whether it succeeded, and continue. + +A chatbot can summarize a bug. An agentic runtime can search the repository, open the relevant files, patch the code, run tests, inspect failures, and retry. + +A chatbot can describe a plan. An agentic runtime can maintain that plan as live state, distribute pieces to subagents, merge their reports, and verify the result. + +The chatbot paradigm has four major constraints: + +1. **Statelessness** — The model does not persist state on its own. Conversation history is simulated state, not durable memory. +2. **Passivity** — The model cannot act unless an external runtime executes its tool-use intent. +3. **Linear context** — Long sessions eventually collide with the context window unless the runtime compacts, persists, or retrieves memory. +4. **No native self-improvement loop** — The model may produce better answers, but the runtime does not automatically preserve and redeploy better behaviors. + +An AOS addresses these constraints by making orchestration a first-class system responsibility. + +--- + +## 2. Definition: what an Agentic OS is and is not + +An **Agentic Operating System** is not the LLM itself. + +It is the runtime around the LLM that provides: + +- an execution loop, +- tool dispatch, +- state and memory management, +- process/subagent orchestration, +- permission boundaries, +- tracing and profiling, +- persistence and recovery, +- evaluation and scoring, +- and, eventually, self-modification. + +A concise definition: + +> An Agentic Operating System is a runtime that turns stateless model calls into persistent, tool-using, multi-process, self-profiling, self-improving computational agents. + +The LLM supplies cognition. The AOS supplies embodiment. + +```mermaid +flowchart LR + U[Human intent] --> AOS[Agentic Operating System] + AOS --> LLM[LLM / reasoning engine] + LLM --> AOS + AOS --> Tools[Tools, files, shell, browser, MCP] + Tools --> AOS + AOS --> Memory[Memory, trace, trajectory] + Memory --> AOS + AOS --> Result[Verified accomplishment] +``` + +--- + +## 3. Codegraff as an AOS case study + +Codegraff's design maps naturally onto operating-system concepts. + +| Traditional OS concept | Agentic OS concept | Codegraff example | +|---|---|---| +| Kernel | Agent execution loop | `Agent.runTurn` | +| Process | Agent instance | root agent or subagent | +| Child process | Isolated worker agent | `subagent`, `workflowTask` | +| Syscall | Tool call | `bash`, `read_file`, `edit_file`, MCP tools | +| Device driver | Tool/MCP adapter | built-in tools, `mcp.zig` | +| Scheduler | Workflow/fanout orchestration | `workflow`, parallel subagents | +| Memory manager | Context/history manager | compaction, session save/resume | +| Filesystem | Durable context substrate | repo files, `.harness/`, trace files | +| Profiler | Execution trace | `harness.trace.jsonl` | +| Process tree | Agent trajectory | `harness.trajectory.jsonl` | +| Program binary | Agent genome | system prompt / persona prompt | +| Package update | Fleet elite pull | `pullElites` | + +This framing matters because it gives us a vocabulary for improving agent systems. Instead of asking only “is the model smart enough?”, we can ask: + +- Is the scheduler good? +- Are tool results represented cleanly? +- Is memory stable and cheap? +- Are failures observable? +- Can the agent recover from transport errors? +- Can better policies be evaluated and promoted? +- Can we evolve specialized behaviors per niche and model class? + +Those are operating-system questions. + +--- + +## 4. The cognitive kernel + +The kernel of an AOS is the loop that repeatedly gives the model state, receives intent, executes that intent, observes the result, and continues until the task is complete. + +In Codegraff, this is structurally the `Agent` loop: + +1. Build a provider-specific request body from the current message history and tool schema. +2. Send the request to the model provider. +3. Parse the response. +4. If the response is final text, complete the turn. +5. If the response contains tool calls, execute them. +6. Append tool results to history. +7. Re-enter the loop. + +```mermaid +sequenceDiagram + participant User + participant Kernel as AOS Kernel / Agent.runTurn + participant Model as LLM Provider + participant Tools as Tool Layer + participant Trace as Trace / Trajectory + + User->>Kernel: objective + Kernel->>Trace: record user turn + + loop until final answer + Kernel->>Model: current state + tool schema + Model-->>Kernel: text or tool calls + Kernel->>Trace: record API call + + alt tool calls requested + Kernel->>Tools: execute selected tools + Tools-->>Kernel: observations / errors + Kernel->>Trace: record tool calls + Kernel->>Kernel: append observations to history + else final response + Kernel-->>User: result + end + end +``` + +This is why a single visible “answer” may require many hidden model calls. The user experiences one task. The kernel executes a state machine. + +--- + +## 5. Processes, subagents, and workflows + +A chatbot has one stream of thought. An AOS can spawn workers. + +In Codegraff, a subagent is not a magical separate species. It is the same `Agent` abstraction instantiated with: + +- a fresh arena, +- an empty history, +- a self-contained task prompt, +- an optional system-prompt override, +- inherited provider/client configuration, +- and restricted capabilities. + +That gives Codegraff a process model. + +```mermaid +graph TD + Root[Root agent] --> T1[Tool call] + Root --> S1[Subagent: reviewer] + Root --> S2[Subagent: researcher] + Root --> S3[Subagent: implementer] + + S1 --> R1[Report: risks / bugs] + S2 --> R2[Report: context / evidence] + S3 --> R3[Report: patch / implementation] + + R1 --> Merge[Root synthesizes] + R2 --> Merge + R3 --> Merge + Merge --> Final[Final answer / action] +``` + +This is a major reason smaller or cheaper models can appear more capable inside a good harness. The model does not need to hold every branch of reasoning in one context. The AOS can externalize work into isolated subprocesses and feed back only the useful reports. + +The tradeoff is overhead: subagents require additional model calls and duplicated context. So the scheduler matters. A good AOS should know when to fan out and when to stay single-threaded. + +--- + +## 6. Tools as peripherals + +An operating system is useful because it mediates access to devices. An Agentic OS is useful because it mediates access to tools. + +For an LLM, a tool call is only a prediction. The runtime must turn that prediction into action. + +Tool handling requires: + +- schema exposure, +- argument parsing, +- permission checks, +- execution, +- timeout handling, +- cancellation, +- result capture, +- error normalization, +- and history reinsertion. + +```mermaid +flowchart TD + Model[Model predicts tool call] --> Parser[Parse and validate arguments] + Parser --> Gate{Permission / policy gate} + Gate -->|allowed| Exec[Execute tool] + Gate -->|needs approval| Human[Ask user] + Human --> Exec + Gate -->|denied| Denied[Return denial observation] + Exec --> Result[Capture stdout / stderr / structured result] + Result --> History[Append observation to agent history] + Denied --> History +``` + +MCP extends this idea by making external tools look like standardized drivers. The AOS does not need to know every possible capability ahead of time. It needs a protocol for discovering and invoking capabilities safely. + +--- + +## 7. Memory architecture + +The LLM's context window is not memory. It is working memory. + +An AOS needs multiple memory tiers: + +| Tier | Role | Example | +|---|---|---| +| L1 working memory | Current prompt/history inside the model context | active messages | +| L2 session memory | Durable state for the current run | `last.session.json` | +| L3 trace memory | Profiling and debugging history | `harness.trace.jsonl` | +| L4 trajectory memory | Evolutionary lineage and scores | `harness.trajectory.jsonl` | +| L5 project/user memory | Cross-session task knowledge | proposed `.harness/memory/` | +| L6 fleet memory | Shared behavioral improvements | promoted personas/elites | + +```mermaid +graph TB + L1[L1: active context] --> C[Compaction] + C --> L2[L2: session summary] + L2 --> Resume[resume / restore] + + Agent[Agent runtime] --> Trace[L3: trace log] + Agent --> Traj[L4: trajectory archive] + Agent --> Project[L5: project memory] + + Traj --> Eval[Evaluation / scoring] + Eval --> Fleet[L6: fleet memory] + Fleet --> Agent +``` + +A key design rule from Codegraff's memory docs is that memory should not mutate the cached prefix every turn. Retrieved memory should enter as append-only context, usually through a tool result or user-turn appendage. That preserves provider KV-cache efficiency and avoids making every turn pay for a changing system prompt. + +--- + +## 8. Tracing, profiling, and self-debugging + +An AOS must be observable. If the agent is a process, then we need process telemetry. + +Codegraff writes a session trace to `harness.trace.jsonl`. The trace records API calls, tool calls, latency, byte counts, context token counts, and errors. This lets the agent inspect its own execution behavior. + +That creates a feedback loop: + +```mermaid +sequenceDiagram + participant Agent + participant Trace as harness.trace.jsonl + participant User + + Agent->>Trace: record API/tool events + User->>Agent: why did that stall? + Agent->>Trace: read recent events + Trace-->>Agent: latencies, result sizes, errors + Agent-->>User: diagnosis + fix +``` + +This is an important AOS property: the runtime gives the model evidence about its own behavior. Without that, the model can only guess why it failed. + +A practical note: trace integrity matters. If trace files become malformed or binary, the self-debugging loop fails. For an AOS, logs are not auxiliary. They are part of the cognitive substrate. + +--- + +## 9. The evolutionary layer: Darwin-Gödel agents + +The most interesting extension of the AOS idea is self-improvement. + +Traditional neural evolution, such as NEAT, evolves weights and topology. That becomes expensive at modern model scale because model weights are huge and evaluation is costly. + +An Agentic OS can move evolution to a cheaper layer: **the behavioral program around the model**. + +In Codegraff, the simplest genome is the system prompt or persona prompt. A prompt can be fingerprinted, archived, mutated, evaluated, scored, and promoted. + +```mermaid +graph LR + Seed[Seed persona / prompt] --> Mutate[Mutate prompt] + Mutate --> Child[Run child agent] + Child --> Judge[Replay judge / evaluator] + Judge --> Score[Signed score] + Score --> Archive[Trajectory archive] + Archive --> Select[Parent selection] + Select --> Mutate +``` + +The Darwin-Gödel framing has two sides: + +- **Darwinian search:** generate behavioral variants and keep the ones that perform better. +- **Gödelian self-modification:** make improvements explicit, durable, and redeployable as part of the agent's own operating logic. + +The crucial move is that we are not evolving billions of weights. We are evolving compact, high-level behavioral policies. + +--- + +## 10. Fleet evolution and MAP-Elites + +A single machine can evolve prompts locally, but a fleet can do something stronger. Every installation can generate and score variants during normal use. A central worker can aggregate only the cheap metadata: prompt hash, score, niche, provider class, eval-set hash, and signed provenance. + +The expensive inference stays with each user's provider. The central system does ranking, not reasoning. + +```mermaid +graph TD + I1[Install A] --> S1[Signed score + genome hash] + I2[Install B] --> S2[Signed score + genome hash] + I3[Install C] --> S3[Signed score + genome hash] + + S1 --> Worker[Fusion worker] + S2 --> Worker + S3 --> Worker + + Worker --> Group[Group by niche × provider_class × eval_set_hash] + Group --> Rank[Rank with confidence / LCB] + Rank --> Promote[Promote elite] + Promote --> Pull[Clients pull elite at startup] + Pull --> Better[Better built-in persona] +``` + +MAP-Elites prevents one global prompt from dominating every use case. Instead, the archive keeps separate champions per behavioral niche and model tier. + +Examples of niches: + +- reviewer, +- researcher, +- implementer, +- skeptic, +- debugging specialist, +- documentation writer, +- benchmark runner. + +This is the AOS equivalent of speciation in NEAT: protect useful specialization instead of forcing all innovations to compete in one undifferentiated population. + +--- + +## 11. NEAT to AOS: the conceptual bridge + +The NEAT paper is useful here because it explains why open-ended improvement needs more than naive mutation. + +| NEAT idea | AOS / Codegraff analogue | +|---|---| +| Genome | System prompt / persona / behavioral policy | +| Historical marking | Prompt fingerprint / lineage hash | +| Mutation | Prompt rewrite or policy change | +| Crossover | Future recombination of structured prompt/policy sections | +| Speciation | Niches, provider classes, eval-set cells | +| Fitness | Replay judge, tests, task success, efficiency | +| Innovation protection | Do not promote from one lucky score; aggregate by cell and confidence | +| Minimal start | Seed prompt, then incremental behavioral additions | +| Complexification | Evolve richer tool policies, memory policies, and workflows | + +The compute advantage comes from moving evolution upward. Instead of searching over model weights, the AOS searches over instructions, workflows, memory designs, and tool-use strategies. + +--- + +## 12. Failure modes and design cautions + +The AOS framing is powerful, but it also exposes real failure modes. + +### 12.1 Tool-output overload + +A broad search can return hundreds of thousands of bytes. If the runtime dumps all of that into context, the next model call may become slow, expensive, or unstable. A good AOS needs output caps, summarization, and search discipline. + +### 12.2 Weak or missing eval identity + +Evolution only works if scores are comparable. A score without a stable `eval_set_hash` is hard to aggregate. Fleet promotion should prefer pinned, deterministic evaluation suites over vague organic success signals. + +### 12.3 Prompt-only genomes may plateau + +Prompt mutation is a good base case, but eventually the genome should become structured: + +```text +genome = + prompt policy + tool-use policy + memory design + decomposition policy + verification policy + communication policy +``` + +Structured genomes make crossover and targeted mutation easier. + +### 12.4 Logs are part of cognition + +If traces are missing, malformed, or unreadable, the agent loses self-observation. Observability is not just a developer feature; it is part of the runtime's intelligence loop. + +### 12.5 The scheduler can waste intelligence + +Subagents are powerful but not free. Over-fanout burns tokens. Under-fanout overloads the root context. Scheduling policy is one of the core optimization surfaces for an AOS. + +--- + +## 13. Roadmap: from prompt evolution to full AOS evolution + +The base case is prompt evolution. The broader target is runtime evolution. + +A staged roadmap: + +1. **Prompt genome** — evolve system prompts/personas. +2. **Pinned evaluation** — require stable eval identities for promotable scores. +3. **Structured prompt genome** — split prompts into behavior sections. +4. **Tool policy genome** — evolve when and how tools are used. +5. **Memory design genome** — evolve retrieval/update policies. +6. **Scheduler genome** — evolve when to spawn subagents or workflows. +7. **Crossover** — recombine successful policies across lineages. +8. **Fleet promotion** — promote reliable elites per niche/model/eval cell. +9. **Self-hosted improvement** — let agents propose, test, and deploy runtime changes under strict evaluation gates. + +The long-term goal is not merely an agent that answers better. It is an execution environment that improves how agents are constructed. + +--- + +## 14. Conclusion + +The Agentic Operating System is the missing abstraction between raw model intelligence and real digital work. + +A model can reason. An AOS can execute. + +A model can propose. An AOS can verify. + +A model can remember within a context window. An AOS can persist, compact, retrieve, and resume. + +A model can be prompted. An AOS can evolve prompts, policies, workflows, and memory designs over time. + +This reframes the central question. The future is not only about larger models. It is also about better runtimes around models: kernels for cognition, schedulers for subagents, drivers for tools, filesystems for memory, profilers for self-observation, and evolutionary mechanisms for improvement. + +In that sense, Codegraff is not just a coding assistant. It is a prototype of an Agentic Operating System: a runtime where LLM cognition becomes persistent, inspectable, executable, and eventually self-improving. diff --git a/benchmarks/README.md b/benchmarks/README.md index 8bec01b5..fc43521a 100644 --- a/benchmarks/README.md +++ b/benchmarks/README.md @@ -22,7 +22,7 @@ task, the model, and the network. | `cost.py` | USD per task (each tool's own usage, [gateway prices](https://codegraff.com/docs/models)) | graff/deepseek-v4-pro **$0.022** vs Claude Code/Opus-4.8 **$0.51** vs Codex/gpt-5.5 **$0.42** | | `memory.py` | Peak RSS + CPU (`/usr/bin/time -l`) | graff **~25 MB** floor vs Node **~410 MB** vs Rust **~206 MB** | | `latency.py` | One-shot turn latency, same gpt-5.5 endpoint, concurrent pairs | graff **4.4 s** vs codex **8.9 s** (one-shot only) | -| `memory_session.py` | RSS-vs-turn slope of ONE persistent `--json` session (the #124 leak detector; `memory.py` is one-shot peak RSS and structurally blind to per-turn growth) | flat slope after warmup = no per-turn leak | +| `memory_session.py` | RSS-vs-turn slope plus session/scratch arena capacity for ONE persistent `--json` session (the #124 leak detector; `memory.py` is one-shot peak RSS and structurally blind to per-turn growth) | flat slope after warmup = no per-turn leak | ```sh python3 benchmarks/cost.py @@ -31,6 +31,11 @@ python3 benchmarks/latency.py 8 python3 benchmarks/memory_session.py 50 # N turns, one persistent session ``` +`memory_session.py` drives and samples one turn at a time so the child remains +alive at every RSS sample. Set `GRAFF_BENCH_MODEL` to test a model other than +`deepseek-v4-pro`; the script enables `GRAFF_MEM_DEBUG` itself and prints both +arena capacities beside process RSS. + Tasks live in `tasks.py` (edit to test other work). ## Honest caveats (please read) diff --git a/benchmarks/memory_session.py b/benchmarks/memory_session.py index 8e4df3fd..25f12540 100644 --- a/benchmarks/memory_session.py +++ b/benchmarks/memory_session.py @@ -13,6 +13,7 @@ Run: CODEGRAFF_API_KEY=... python3 benchmarks/memory_session.py [N] [graff-path] N defaults to 16; graff-path defaults to `graff` on PATH. + GRAFF_BENCH_MODEL selects the model (default: deepseek-v4-pro). """ import subprocess, json, os, sys @@ -20,6 +21,8 @@ N = int(sys.argv[1]) if len(sys.argv) > 1 else 16 GRAFF = sys.argv[2] if len(sys.argv) > 2 else "graff" +MODEL = os.environ.get("GRAFF_BENCH_MODEL", "deepseek-v4-pro") +SLOPE_LIMIT_MB = 0.1 # Short prompts that each stream a paragraph — many SSE events per turn (the # parse-garbage axis #124 is about) without slow multi-tool work, so the RSS @@ -44,40 +47,53 @@ def rss_mb(pid): def main(): - env = {**os.environ, "GRAFF_NO_TELEMETRY": "1", "GRAFF_FLEET": "off"} - p = subprocess.Popen([GRAFF, "--model", "deepseek-v4-pro", "--json"], + if N < 2: + raise SystemExit("N must be at least 2") + env = {**os.environ, "GRAFF_NO_TELEMETRY": "1", "GRAFF_FLEET": "off", + "GRAFF_MEM_DEBUG": "1"} + p = subprocess.Popen([GRAFF, "--model", MODEL, "--json"], stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, text=True, env=env, bufsize=1) samples = [] try: - # Queue all turns up front (small JSON lines fit the pipe buffer), keep - # stdin open so the child stays alive for the final RSS sample, then read - # + sample at each turn boundary. graff processes them sequentially. + # Drive one turn at a time and keep stdin open through every sample. + # Closing it up front races the final sample against process exit, which + # can produce a bogus 0 MB reading and a wildly negative slope. for i in range(N): p.stdin.write(json.dumps({"type": "user", "text": PROMPTS[i % len(PROMPTS)]}) + "\n") - p.stdin.flush() - p.stdin.close() # EOF after the queued turns so graff drains them all then exits - done = 0 - while done < N: - line = p.stdout.readline() - if not line: - raise RuntimeError("graff exited early after %d/%d turns" % (done, N)) - try: - ev = json.loads(line) - except Exception: - continue - if ev.get("type") == "error": - print(" (error: %s)" % ev.get("message")) - if ev.get("type") == "turn" and ev.get("complete"): - done += 1 - r = rss_mb(p.pid) - samples.append((done, r)) - print(" turn %2d: RSS = %7.1f MB" % (done, r)) + p.stdin.flush() + arena = None + while True: + line = p.stdout.readline() + if not line: + raise RuntimeError("graff exited early after %d/%d turns (exit %s)" % + (i, N, p.poll())) + try: + ev = json.loads(line) + except Exception: + continue + if ev.get("type") == "error": + raise RuntimeError("turn %d failed: %s" % (i + 1, ev.get("message"))) + if ev.get("type") == "mem": + arena = (ev.get("session_arena_kb", 0), ev.get("scratch_arena_kb", 0)) + if ev.get("type") == "turn" and ev.get("complete"): + r = rss_mb(p.pid) + if r <= 0: + raise RuntimeError("could not sample RSS for live graff process %d" % p.pid) + samples.append((i + 1, r)) + detail = (" arenas: session=%d KB scratch=%d KB" % arena) if arena else "" + print(" turn %2d: RSS = %7.1f MB%s" % (i + 1, r, detail)) + break finally: - try: - p.stdin.close(); p.terminate(); p.wait(timeout=5) - except Exception: - p.kill() + if p.stdin and not p.stdin.closed: + p.stdin.close() + if p.poll() is None: + p.terminate() + try: + p.wait(timeout=5) + except subprocess.TimeoutExpired: + p.kill() + p.wait() tail = samples[2:] if len(samples) > 4 else samples # drop warmup if len(tail) >= 2: @@ -87,7 +103,12 @@ def main(): mx, my = sum(xs) / n, sum(ys) / n den = sum((x - mx) ** 2 for x in xs) or 1.0 slope = sum((x - mx) * (y - my) for x, y in tail) / den - verdict = "FLAT — no per-turn leak" if abs(slope) < 0.1 else "GROWING — leak present" + if slope > SLOPE_LIMIT_MB: + verdict = "GROWING — investigate retained memory" + elif slope < -SLOPE_LIMIT_MB: + verdict = "SHRINKING — no per-turn leak" + else: + verdict = "FLAT — no per-turn leak" print("\nRSS growth over turns %d..%d: %+.3f MB/turn (%s)" % (xs[0], xs[-1], slope, verdict)) print("first-tail %.1f MB -> peak %.1f MB" % (ys[0], max(ys))) diff --git a/graff-dgm.md b/graff-dgm.md new file mode 100644 index 00000000..5dcd0656 --- /dev/null +++ b/graff-dgm.md @@ -0,0 +1,166 @@ +# graff DGM — federated fleet evolution (read me) + +A map of how graff's self-improvement loop works end-to-end, what's wired right +now, the two repos it spans, and how to operate it. Working note for you — not +committed, not published. + +> One line: **thousands of installs generate + score prompt-persona variants +> during normal use; a central worker ranks them per (niche × model-tier × +> eval-suite) cell over signed scores with no central inference; the winner ships +> back to everyone as a better built-in persona.** Single-tenant DGM → federated +> DGM. Full design: `docs/hyperagents.md §9`. + +--- + +## The loop (A->E) + +| phase | who runs it | what happens | +|---|---|---| +| **A - Harvest** | every install (fleet on) | signed pinned-eval scores + variant fingerprints accumulate in D1 | +| **B - Propose** | the user's own provider | mine local failures -> mutate a niche's elite -> run the eval -> sign(score, eval_set_hash, provider_class) -> **submit** (the genome text rides along on `propose`) | +| **C - Rank** | central worker (SQL, no inference) | per cell: HMAC-verify, group by (niche x provider_class x eval_set_hash), K-install floor, Wilson LCB | +| **D - Promote** | central worker (cron / `POST /v1/promote`) | best-LCB candidate beats the incumbent by a margin -> flip `live=1` in `harness_elites`, emit `fleet:promote` | +| **E - Distribute** | graff startup | graff GETs `/v1/elites` for its tier; `loadAgentTypes` prefers a live champion's prompt over the baked built-in | + +The load-bearing rule: **"let the provider be me."** Every expensive step +(mutating, evaluating) runs on each user's own provider during their session. +Central infra does only cheap deterministic SQL. No central inference -> it runs +at fleet scale for free. **Task/user content never leaves the machine** - only +the genome (persona prompt text), the score float, `eval_set_hash`, +`provider_class`, and the deterministic eval artifact. + +--- + +## Where the code lives (two repos) + +| piece | repo / path | branch | +|---|---|---| +| **Harness** (CLI) - emits the signals, pulls elites | `~/codegraff` / `src/main.zig` | (local working tree) | +| **Backend** (Cloudflare worker) - ingest, stats, rank, promote, serve elites | `~/zigrepper/services/harness-telemetry` | `feat/agent-archive-and-tier` | + +The worker shares D1 (`codegraff-db`) with the trajectory/agent archive. `main` +of zigrepper is **pre-fleet**; all the fleet work is on +`feat/agent-archive-and-tier` (that's what's deployed). Backend edits in this +pass were made on an isolated **worktree** at `/tmp/htel-fleet-wt` so the +zigrepper `main` checkout (with its uncommitted `codedbpro/` changes) was never +touched. + +--- + +## What's wired now + +### Harness (`src/main.zig`) +- `Telemetry.fleetEvent(...)` -> OTLP `body:"fleet"` record with a `kind` attr - + **propose** (variant spawn, in `runSub`), **submit** (score write-back), + **elite_pull** (startup). Matches the SDK `_fleet_signal` byte-for-byte. +- `providerClass(model)` -> `frontier | mid | small`: a curated model-family + table, falling back to the models.dev output-price graff already carries. +- `pullElites(...)` at startup -> GET `/v1/elites?provider_class&eval_set_hash`, + emit `fleet:elite_pull`, override a niche's prompt with a live champion. +- **Genome-send (new):** `fleet:propose` now carries `prompt_text` (the persona + genome = the subagent's `sys_override`), capped at 8 KB. This is what lets the + backend ship a champion's prompt later. Submit/elite_pull send no text (the + genome is already stored, keyed by `prompt_sha`). +- `GRAFF_FLEET=off` env + `/fleet [on|off]` REPL command - a fleet opt-out + independent of the telemetry opt-out. Gates `fleetEvent` + `pullElites`. + +### Backend (`services/harness-telemetry/src/index.ts`) +- `POST /v1/logs` (OTLP ingest, queue-buffered) -> `unpack`: `session`-> + `harness_sessions`, `score`->`harness_scores` (with `provider_class`, + `eval_set_hash`, `niche`), everything else -> `harness_events`. +- **Genome capture (new):** a `fleet` record carrying `prompt_text` upserts + `harness_genomes(prompt_sha -> text)` - deduped, so the genome rides the wire + once per variant, not once per score. +- `GET /v1/stats` -> totals + per-suite leaderboards + the `fleet` block + (`proposes/submits/elite_pulls/promotes/live_cells`). +- `GET /v1/elites` -> live champions for a tier; falls back to best-mean + candidate per niche over the K-install floor when nothing's promoted yet. +- **Promote pass (new) - `runPromote()`:** the piece that was missing. Per cell, + over a trailing 7-day window, joins `harness_scores` -> `harness_genomes` + (only genomes we hold can ship), applies the K-install floor, ranks by + **Wilson LCB** of the mean score, and flips the winner `live=1` if it beats + the incumbent's LCB by the margin. Emits `fleet:promote`. Triggered by a + **cron** (`triggers.crons: ["0 */6 * * *"]`) and an admin **`POST /v1/promote`**. + +### Data model (D1) +- `harness_scores` - one row per score; carries `provider_class`, + `eval_set_hash`, `niche` (migration `0005`). +- `harness_genomes` - `prompt_sha -> prompt_text` (migration **`0006`**, new). +- `harness_elites` - one row per grid cell; `live=1` is the shipped champion. +- `harness_fleet_fitness` - view rolling scores up per cell with a distinct- + install count (the K-floor input). + +Knobs (in `index.ts`): `K_INSTALLS=3`, `MARGIN=0.02`, `WINDOW_DAYS=7`, `Z=1.96`. + +--- + +## How to operate it + +```bash +# Backend (on the worktree, or after merging feat/agent-archive-and-tier) +cd ~/zigrepper/services/harness-telemetry +npm run migrate:remote # applies 0003..0006 (adds harness_genomes) +npm run deploy # publishes the worker incl. the cron + /v1/promote +curl -XPOST https:///v1/promote # manual promote pass (or wait for cron) + +# Harness - opt out of fleet contribution for a run/session +GRAFF_FLEET=off graff ... # or /fleet off in the REPL +``` + +`POST /v1/promote` returns `{ evaluated, promoted: [...] }`. After a promote, +`/v1/stats` `promotes`/`live_cells` move and `/v1/elites` returns +`source:"promoted"` with `prompt_text`, which graff then prefers at startup. + +--- + +## What actually makes it evolve (current state) + +The signals flow and the counters climb, but a champion only emerges once the +feedback is **groupable + repeated**: + +1. **Pin `eval_set_hash` on every score.** Today organic scores are often + `eval_set_hash: null` -> they never form a cell. Route scoring through a fixed + suite (wire `--eval`/`--judge` to stamp a stable hash). *This is the remaining + harness lever.* +2. **Score each genome more than once** so the Wilson LCB has data (n=1 is + meaningless). +3. With (1)+(2), the 24+ installs accumulate per cell -> cross the K-floor -> + `runPromote` flips a champion -> it ships fleet-wide. + +Order of effect: pin the hash -> cells fill -> LCB tightens -> promote -> the +baked persona is superseded for that (niche, tier). + +--- + +## Defense stack (why the backend can decide unattended) + +1. **Signed, artifact-bound scores** - HMAC over the canonical message + (Step 0); with `SCORE_KEY` set, unsigned/forged rows are dropped on ingest. +2. **Sybil floor** - a cell promotes only with >=K distinct `install_id`s. +3. **LCB, not mean** - a 1.0 from 3 installs loses to a 0.95 from 4,000. +4. **Genome-gated** - only a genome we actually hold (`harness_genomes`) can be + promoted, so a champion always has shippable text. +5. **Reversible** - what ships is *a prompt*; demote = flip `live=0`. Keep a + "last N champions" changelog for one-tap revert. + +--- + +## Tests + +- **Harness unit** (`zig build test`): `providerClass` tiering + fleet OTLP + serialization. +- **Keyless e2e** (`scripts/e2e_fleet.sh` + `scripts/otlp_mock.py`): drives the + real binary against a capturing collector, asserts the emitted records. +- **Backend**: `tsc --noEmit` clean. For a live promote test: `npm run + migrate:local` + `wrangler dev`, seed >=3 installs of one pinned-hash genome, + `POST /v1/promote`, check `/v1/elites` returns `source:"promoted"`. + +--- + +## Status of this pass + +- Harness genome-send: **done**, builds + tests green (`~/codegraff`, uncommitted). +- Backend promote pass + genome store + cron + `0006` migration: **done**, + `tsc` clean (`/tmp/htel-fleet-wt`, uncommitted - review, then merge + `feat/agent-archive-and-tier` and deploy). +- Nothing committed or deployed. Remaining harness lever: pin `eval_set_hash`. diff --git a/phase2-escalation.md b/phase2-escalation.md new file mode 100644 index 00000000..bf23f8c2 --- /dev/null +++ b/phase2-escalation.md @@ -0,0 +1,114 @@ +# Phase 2, slice 1: Dynamic escalation / de-escalation (design) + +Working note, not committed. First slice of vision.md Phase 2 (the Smart +Router). Scope: failure-triggered tier promotion at subagent boundaries. +Explicitly NOT in this slice: mid-turn model switching, complexity +profiling, the cache-switching predictor. + +## Why this slice first + +- It is the smallest closed loop: run -> detect failure -> rerun one tier + up -> score both -> emit the trace. +- It sidesteps the Cache-Switching Paradox entirely. A subagent respawn is + a fresh context anyway, so there is no cache momentum to lose. Mid-turn + switching (the hard economic problem) waits until we have data. +- Its traces ARE the training data for the rest of Phase 2. The predictor + needs (task, tier, outcome, cost) tuples; nothing produces those today. + +## What exists to build on (all in src/main.zig unless noted) + +| piece | where | role in this slice | +|---|---|---| +| `runSub(ctx, kind, label, prompt, sys_override, niche)` | ~L15900 | the seam: every subagent + workflow task passes through it | +| `providerClass(model)` -> frontier/mid/small | main.zig (19 refs) | the tier ladder | +| pricing catalog | `src/pricing.zig` | cost of a retry, cost delta in the trace | +| fitness scores + `scoreVariants` | main.zig | the success/failure signal for eval'd runs | +| `Telemetry.fleetEvent` (propose/submit/elite_pull) | main.zig | pattern to copy for the new `escalate` event | +| `/effort`, `/fast`, `/fleet` REPL toggles | main.zig / repl.zig | pattern for `/escalate on|off` | + +## Design + +### Failure signal (v1, cheap and unambiguous) + +A subagent run counts as FAILED when any of: +1. the run errored (tool loop died, provider error after retries); +2. an eval/judge score exists for the run and is below `ESCALATE_FLOOR` + (default 0.5, matches the fleet score scale); +3. the run hit the turn cap without producing a result. + +No LLM-judge-of-everything in v1. If nothing scored the run and it did +not error, it is a success. Keep the signal deterministic so traces are +trustworthy. + +### Escalation rule + +- Ladder: small -> mid -> frontier (reuse `providerClass` ordering). +- On FAILED and tier < frontier and escalation enabled: rerun the same + `runSub` args once at the next tier up. One rung per failure, max one + retry per subagent (a frontier failure is just a failure). +- The rerun replaces the failed result for the caller; the failed + attempt still scores/submits as itself (its genome earned its score). + +### De-escalation rule (shadow, cheap) + +- Per (niche, tier): after `DEESCALATE_STREAK` (default 5) consecutive + successes at a tier, mark the niche "try one tier down next time". +- The downgraded run is live, not shadowed (shadowing doubles cost, which + is the thing we are minimizing). If it fails, the escalation rule + already catches it and the streak resets. Worst case cost = one cheap + failed attempt + the escalated rerun. + +### Telemetry: the `escalate` fleet event + +New `fleetEvent` kind `escalate`, emitted on every escalation AND every +deliberate de-escalated run: + +``` +{ kind: "escalate", niche, from_class, to_class, reason: "error"|"score"|"cap"|"deescalate", + failed_score, rerun_score, cost_delta_usd, ms_delta } +``` + +Same privacy posture as the rest of the fleet loop: no task content, no +prompt text; scores + tiers + deltas only. Backend just needs an events +passthrough (harness_events already captures unknown kinds; no migration). + +### Controls + +- `GRAFF_ESCALATE=off` env + `/escalate [on|off]` REPL command, mirroring + the `/fleet` pattern. Default: ON for subagents (it only ever spends + money to rescue a failure), OFF for de-escalation until traces say the + streak heuristic is safe. +- Knobs as consts next to the fleet knobs: `ESCALATE_FLOOR=0.5`, + `DEESCALATE_STREAK=5`. + +## Implementation order + +1. Tier ladder helper: `nextClassUp(class)` / `nextClassDown(class)` + + pick a concrete model for a class (cheapest in class from pricing.zig; + respects the user's provider config). +2. Wrap the `runSub` call sites (workflowTask + the Task tool path) with + the retry-one-rung-up loop. Keep `runSub` itself untouched. +3. `fleetEvent("escalate", ...)` + the env/REPL toggles. +4. Unit tests: ladder ordering, failure classification, one-rung cap. + E2e: extend `scripts/e2e_fleet.sh` mock to assert an escalate record + when a forced-fail subagent reruns. +5. (later, own slice) de-escalation streak tracking, persisted per niche + in the session state file. + +## Exit criteria for the slice + +- A forced-failure subagent on a small-class model visibly reruns on mid, + the caller gets the mid result, and the collector sees one `escalate` + record with both scores and the cost delta. +- `zig build test` green; e2e green; `GRAFF_ESCALATE=off` produces + byte-identical behavior to today. + +## What the traces unlock next + +- Cache-Switching Predictor (vision.md P2): needs P(success | tier, + niche) priors -> read them off accumulated escalate records per cell. +- Complexity profiling: label = "did small tier succeed"; the escalate + stream is the labeled dataset, no separate collection pass. +- Intelligence arbitrage (P3): success-per-dollar per (niche, tier) falls + out of rerun_score / cost_delta aggregation, SQL only, same + no-central-inference rule as the DGM loop. diff --git a/scripts/check-zig-lines.sh b/scripts/check-zig-lines.sh new file mode 100755 index 00000000..89db51e1 --- /dev/null +++ b/scripts/check-zig-lines.sh @@ -0,0 +1,25 @@ +#!/usr/bin/env bash +set -euo pipefail + +repo_root=$(git -C "$(dirname "${BASH_SOURCE[0]}")/.." rev-parse --show-toplevel) +cd "$repo_root" + +# Keep every checked-in Zig source below the agreed 700-line ceiling. Include +# untracked, non-ignored files locally so a newly split module is checked before +# it is staged; CI sees the same files once they are committed. +max_lines=699 +failed=0 + +while IFS= read -r -d '' file; do + lines=$(awk 'END { print NR }' "$file") + if (( lines > max_lines )); then + printf 'Zig source exceeds %d lines: %s (%d)\n' "$max_lines" "$file" "$lines" >&2 + failed=1 + fi +done < <(git ls-files -z --cached --others --exclude-standard -- '*.zig') + +if (( failed != 0 )); then + exit 1 +fi + +printf 'All Zig sources are at most %d lines.\n' "$max_lines" diff --git a/scripts/codex_ws_mock.py b/scripts/codex_ws_mock.py index ca8170ac..da3610ce 100644 --- a/scripts/codex_ws_mock.py +++ b/scripts/codex_ws_mock.py @@ -35,6 +35,7 @@ import threading import time from dataclasses import dataclass, field +from typing import Callable WS_GUID = "258EAFA5-E914-47DA-95CA-C5AB0DC85B11" @@ -71,11 +72,14 @@ def assistant_item() -> dict: } -def turn_events() -> list[dict]: +def turn_events(response_id: str = "resp_mock_1") -> list[dict]: """The two events one mocked turn produces, in send order.""" return [ {"type": "response.output_item.done", "item": assistant_item()}, - {"type": "response.completed", "response": {"usage": dict(USAGE)}}, + { + "type": "response.completed", + "response": {"id": response_id, "usage": dict(USAGE)}, + }, ] @@ -86,6 +90,20 @@ class Frame: fin: bool +@dataclass(frozen=True) +class RecordedRequest: + """One decoded Responses request, in server-observed order. + + ``connection_id`` is populated for WebSocket requests so tests can prove a + post-compaction request re-anchored on a new socket. SSE requests use None. + """ + + ordinal: int + transport: str + connection_id: int | None + body: dict + + class _SockReader: """Buffered exact-read wrapper so bytes recv'd past a boundary (e.g. the HTTP head) still feed later frame/body reads.""" @@ -169,7 +187,9 @@ def _read_message(reader: _SockReader, sock: socket.socket) -> Frame: raise ConnectionError("message too long") parts.append(frame.payload) if frame.fin: - return Frame(opcode if opcode is not None else OP_TEXT, b"".join(parts), True) + return Frame( + opcode if opcode is not None else OP_TEXT, b"".join(parts), True + ) def ws_accept_value(key: str) -> str: @@ -181,13 +201,22 @@ def ws_accept_value(key: str) -> str: @dataclass class CodexMock: """Threaded fake codex backend: start() binds and returns the real port, - stop() tears it down. ws_turns/sse_turns count served turns per transport.""" + stop() tears it down. ws_turns/sse_turns count served turns per transport. + + ``events_for_request`` optionally scripts replies from the decoded request; + the default remains the fixed mock reply used by the transport smoke tests. + """ host: str = "127.0.0.1" port: int = 0 verbose: bool = False ws_turns: int = 0 sse_turns: int = 0 + ws_connections: int = 0 + events_for_request: Callable[[RecordedRequest], list[dict]] | None = field( + default=None, repr=False + ) + requests: list[RecordedRequest] = field(default_factory=list, repr=False) _sock: socket.socket | None = field(default=None, repr=False) _threads: list[threading.Thread] = field(default_factory=list, repr=False) _lock: threading.Lock = field(default_factory=threading.Lock, repr=False) @@ -217,12 +246,32 @@ def stop(self) -> None: thread.join(timeout=1.0) self._threads.clear() + def recorded_requests(self) -> list[RecordedRequest]: + """Return a stable snapshot for assertions after a scenario completes.""" + with self._lock: + return list(self.requests) + # ── internals ───────────────────────────────────────────────────────── def _log(self, message: str) -> None: if self.verbose: print(f"[codex-mock] {message}", file=sys.stderr, flush=True) + def _events( + self, transport: str, connection_id: int | None, body: dict + ) -> list[dict]: + with self._lock: + request = RecordedRequest( + ordinal=len(self.requests) + 1, + transport=transport, + connection_id=connection_id, + body=body, + ) + self.requests.append(request) + if self.events_for_request is not None: + return self.events_for_request(request) + return turn_events(f"resp_mock_{request.ordinal}") + def _accept_loop(self) -> None: srv = self._sock assert srv is not None @@ -275,7 +324,12 @@ def _handle(self, conn: socket.socket) -> None: pass conn.close() - def _serve_ws(self, conn: socket.socket, reader: _SockReader, headers: dict[str, str]) -> None: + def _serve_ws( + self, conn: socket.socket, reader: _SockReader, headers: dict[str, str] + ) -> None: + with self._lock: + self.ws_connections += 1 + connection_id = self.ws_connections accept = ws_accept_value(headers.get("sec-websocket-key", "")) conn.sendall( ( @@ -286,7 +340,7 @@ def _serve_ws(self, conn: socket.socket, reader: _SockReader, headers: dict[str, "\r\n" ).encode("ascii") ) - self._log("ws upgraded") + self._log(f"ws upgraded (connection {connection_id})") while True: message = _read_message(reader, conn) if message.opcode == OP_CLOSE: @@ -304,17 +358,27 @@ def _serve_ws(self, conn: socket.socket, reader: _SockReader, headers: dict[str, self._log(f"ws <- {etype} ({len(message.payload)}b)") if etype != "response.create": continue - for ev in turn_events(): - _send_frame(conn, OP_TEXT, json.dumps(ev, separators=(",", ":")).encode("utf-8")) + for ev in self._events("ws", connection_id, event): + _send_frame( + conn, OP_TEXT, json.dumps(ev, separators=(",", ":")).encode("utf-8") + ) self._log(f"ws -> {ev['type']}") with self._lock: self.ws_turns += 1 - def _serve_sse(self, conn: socket.socket, reader: _SockReader, headers: dict[str, str]) -> None: + def _serve_sse( + self, conn: socket.socket, reader: _SockReader, headers: dict[str, str] + ) -> None: length = int(headers.get("content-length", "0") or 0) body = reader.read_exact(length) if length else b"" self._log(f"sse <- POST ({len(body)}b)") - events = turn_events() + try: + parsed = json.loads(body.decode("utf-8")) + except (UnicodeDecodeError, ValueError): + parsed = {} + if not isinstance(parsed, dict): + parsed = {} + events = self._events("sse", None, parsed) payload = "".join( f"data: {json.dumps(ev, separators=(',', ':'))}\n\n" for ev in events ).encode("utf-8") @@ -338,7 +402,9 @@ def main() -> None: parser = argparse.ArgumentParser( description="Fake codex Responses backend (WebSocket + SSE) for manual debugging." ) - parser.add_argument("--port", type=int, default=0, help="TCP port to bind (default: ephemeral)") + parser.add_argument( + "--port", type=int, default=0, help="TCP port to bind (default: ephemeral)" + ) args = parser.parse_args() mock = CodexMock(port=args.port, verbose=True) port = mock.start() diff --git a/scripts/test-json-live.py b/scripts/test-json-live.py new file mode 100644 index 00000000..9f2f464f --- /dev/null +++ b/scripts/test-json-live.py @@ -0,0 +1,138 @@ +#!/usr/bin/env python3 +"""Live local-backend proof that --json stdout stays strict JSONL.""" + +import json +import os +import subprocess +import sys +import tempfile + +from codex_ws_mock import CodexMock, REPLY_TEXT + + +_arg = sys.argv[1] if len(sys.argv) > 1 else "graff" +GRAFF = os.path.abspath(_arg) if os.sep in _arg else _arg + + +def decode(line: str) -> dict | None: + if not line.strip(): + return None + try: + event = json.loads(line) + except json.JSONDecodeError as exc: + raise AssertionError(f"non-JSON stdout line: {line!r}") from exc + if not isinstance(event, dict) or not isinstance(event.get("type"), str): + raise AssertionError(f"malformed JSON event: {event!r}") + return event + + +def main() -> None: + mock = CodexMock() + port = mock.start() + try: + with tempfile.TemporaryDirectory(prefix="graff-json-live-") as tmp: + codex_home = os.path.join(tmp, "codex-home") + os.makedirs(codex_home) + with open( + os.path.join(codex_home, "auth.json"), "w", encoding="utf-8" + ) as fh: + json.dump( + { + "tokens": { + "access_token": "json-live-mock", + "account_id": "acct-json-live", + } + }, + fh, + ) + harness_dir = os.path.join(tmp, ".harness") + os.makedirs(harness_dir) + with open( + os.path.join(harness_dir, "settings.json"), + "w", + encoding="utf-8", + ) as fh: + json.dump({"skills": {"codedbpro": False}}, fh) + + env = os.environ.copy() + for name in tuple(env): + if name.startswith("GRAFF_") or name.startswith("CODEX_"): + env.pop(name) + env.update( + { + "HOME": tmp, + "CODEX_HOME": codex_home, + "GRAFF_CODEX_URL": ( + f"http://127.0.0.1:{port}/backend-api/codex/responses" + ), + "GRAFF_FLEET": "off", + "GRAFF_NO_TELEMETRY": "1", + } + ) + process = subprocess.Popen( + [GRAFF, "--model", "codex", "--json", "--no-telemetry"], + cwd=tmp, + env=env, + stdin=subprocess.PIPE, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + text=True, + bufsize=1, + ) + events: list[dict] = [] + assert process.stdin is not None + assert process.stdout is not None + try: + for turn in range(2): + process.stdin.write( + json.dumps({"type": "user", "text": f"live turn {turn}"}) + + "\n" + ) + process.stdin.flush() + while True: + line = process.stdout.readline() + if not line: + stderr = process.stderr.read() if process.stderr else "" + raise AssertionError( + f"graff exited early ({process.poll()}): {stderr}" + ) + event = decode(line) + if event is None: + continue + events.append(event) + if event.get("type") == "turn" and event.get("complete"): + break + process.stdin.close() + for line in process.stdout: + event = decode(line) + if event is not None: + events.append(event) + code = process.wait(timeout=5) + if code != 0: + stderr = process.stderr.read() if process.stderr else "" + raise AssertionError(f"graff exit={code}: {stderr}") + finally: + if process.poll() is None: + process.terminate() + process.wait(timeout=5) + + text_events = [event for event in events if event["type"] == "text"] + turns = [event for event in events if event["type"] == "turn"] + if len(text_events) != 2 or any( + event.get("text") != REPLY_TEXT for event in text_events + ): + raise AssertionError(f"missing unstreamed text events: {text_events!r}") + if len(turns) != 2 or any( + event.get("text") != REPLY_TEXT or not event.get("complete") + for event in turns + ): + raise AssertionError(f"bad terminal turn events: {turns!r}") + if mock.ws_turns != 2: + raise AssertionError(f"expected two WS turns, got {mock.ws_turns}") + finally: + mock.stop() + print("ok live Codex WS emits strict JSONL, including fallback text events") + + +if __name__ == "__main__": + main() diff --git a/scripts/test-pty-codex-ws.py b/scripts/test-pty-codex-ws.py index 630763af..e794a7ac 100644 --- a/scripts/test-pty-codex-ws.py +++ b/scripts/test-pty-codex-ws.py @@ -1,8 +1,12 @@ #!/usr/bin/env python3 -"""Deterministic real-PTY test for codex transport: WebSocket primary, forced -SSE fallback, and GRAFF_CODEX_WS=off — each against a local fake backend -(codex_ws_mock) whose ws_turns/sse_turns counters prove which transport -actually served the turn, so a silent fallback cannot fake a pass.""" +"""Deterministic real-PTY tests for Codex transport and mid-turn compaction. + +The transport smokes cover WebSocket primary, forced SSE fallback, and +GRAFF_CODEX_WS=off. The regression scenarios drive real tool loops whose +server-reported usage crosses compact@ while local history stays tiny. They +prove both successful compaction and transactional rollback after an empty +summary across WS -> quiet SSE compaction -> fresh WS. +""" import json import os @@ -10,18 +14,186 @@ import sys import tempfile -from codex_ws_mock import REPLY_TEXT, CodexMock +from codex_ws_mock import REPLY_TEXT, CodexMock, RecordedRequest from pty_harness import PtySession, terminal_text _arg = sys.argv[1] if len(sys.argv) > 1 else "graff" GRAFF = os.path.abspath(_arg) if os.sep in _arg else _arg -# Mock usage total is 1500; compactTokenCount (src/agent.zig) renders that as -# "1k" (integer division), and 1500 tokens is 0% of any realistic window. The -# window itself tracks the baked codex catalog (pricing.zig), which moves -# between releases — so assert the meter SHAPE plus the compactAt invariant -# (compact = context/10*8, src/provider.zig) instead of a frozen window size. -METER_RE = re.compile(r"1k/(\d+)k ctx \(0% · compact@(\d+)k\)") + +# The reported 1500-token usage is conservatively floored by graff's serialized +# request estimate, so the displayed used count can move with the built-in tool +# schema. Assert the meter shape and invariants rather than freezing either the +# prefill estimate or the Codex catalog window. +METER_RE = re.compile(r"(\d+)(k?)/(\d+)k ctx \((\d+)% · compact@(\d+)k\)") +COMPACTING_RE = re.compile(r"compacting ~(\d+) tokens") + +MIDTURN_PROMPT = "exercise the server-side context meter" +MIDTURN_SUMMARY = "The user asked to exercise the server-side context meter." +MIDTURN_FINAL = "done after mid-turn compact" +# Cross compact@ (80%) but stay below the destructive recovery boundary (95%). +# The smoke scenarios set this from the context meter emitted after the runtime +# Codex catalog has loaded, rather than the static --schema catalog. +MIDTURN_TOTAL_TOKENS = 0 +MIDTURN_CONTEXT_TOKENS = 0 +MIDTURN_REASONING_MARKER = "retained-active-reasoning:" +MIDTURN_REASONING_BYTES = 128 * 1024 + +TRANSACTIONAL_PROMPT = "prove failed compaction keeps the live tool loop" +TRANSACTIONAL_FINAL = "done after transactional compaction failure" +TRANSACTIONAL_REASONING_MARKER = "transactional-active-reasoning:" +TRANSACTIONAL_CALL_ID = "call_transactional_1" + + +def response_events( + item: dict | list[dict], response_id: str, total_tokens: int +) -> list[dict]: + """Build the two Responses events parseResponses consumes.""" + output_tokens = 1_000 if total_tokens > 2_000 else 100 + items = item if isinstance(item, list) else [item] + return [ + *( + {"type": "response.output_item.done", "item": output_item} + for output_item in items + ), + { + "type": "response.completed", + "response": { + "id": response_id, + "usage": { + "input_tokens": total_tokens - output_tokens, + "input_tokens_details": {"cached_tokens": 0}, + "output_tokens": output_tokens, + "total_tokens": total_tokens, + }, + }, + }, + ] + + +def message_item(text: str, item_id: str) -> dict: + return { + "type": "message", + "id": item_id, + "status": "completed", + "role": "assistant", + "content": [{"type": "output_text", "text": text, "annotations": []}], + } + + +def active_reasoning_item() -> dict: + """A large current-loop item that compact() must prune before full resend.""" + encrypted = MIDTURN_REASONING_MARKER + "R" * ( + MIDTURN_REASONING_BYTES - len(MIDTURN_REASONING_MARKER) + ) + return { + "type": "reasoning", + "id": "rs_midturn_1", + "summary": [], + "encrypted_content": encrypted, + } + + +def midturn_events(request: RecordedRequest) -> list[dict]: + """Script tool call -> compaction summary -> final answer.""" + if request.ordinal == 1: + item = { + "type": "function_call", + "id": "fc_midturn_1", + "call_id": "call_midturn_1", + "name": "todo_read", + "arguments": "{}", + "status": "completed", + } + # Real high-effort Responses tool loops return reasoning immediately + # before the function call. It is current-turn history here, but once + # compact() appends its synthetic user turn it must be pruned before the + # full SSE summary resend. + return response_events( + [active_reasoning_item(), item], + "resp_midturn_1", + MIDTURN_TOTAL_TOKENS, + ) + if request.ordinal == 2: + return response_events( + message_item(MIDTURN_SUMMARY, "msg_midturn_summary"), + "resp_midturn_summary", + 1_100, + ) + return response_events( + message_item(MIDTURN_FINAL, "msg_midturn_final"), + f"resp_midturn_{request.ordinal}", + 1_300, + ) + + +def transactional_reasoning_item() -> dict: + encrypted = TRANSACTIONAL_REASONING_MARKER + "R" * ( + MIDTURN_REASONING_BYTES - len(TRANSACTIONAL_REASONING_MARKER) + ) + return { + "type": "reasoning", + "id": "rs_transactional_1", + "summary": [], + "encrypted_content": encrypted, + } + + +def transactional_events(request: RecordedRequest) -> list[dict]: + """Script tool call -> empty summary -> answer from restored live history.""" + if request.ordinal == 1: + call = { + "type": "function_call", + "id": "fc_transactional_1", + "call_id": TRANSACTIONAL_CALL_ID, + "name": "todo_read", + "arguments": "{}", + "status": "completed", + } + return response_events( + [transactional_reasoning_item(), call], + "resp_transactional_1", + MIDTURN_TOTAL_TOKENS, + ) + if request.ordinal == 2: + # A syntactically valid Responses answer with no summary text exercises + # compact()'s EmptySummary rollback, rather than a transport failure. + return response_events( + message_item("", "msg_transactional_empty_summary"), + "resp_transactional_empty_summary", + 1_100, + ) + return response_events( + message_item(TRANSACTIONAL_FINAL, "msg_transactional_final"), + f"resp_transactional_{request.ordinal}", + 1_300, + ) + + +def user_text(item: object) -> str | None: + if not isinstance(item, dict) or item.get("role") != "user": + return None + content = item.get("content") + return content if isinstance(content, str) else None + + +def assert_compaction_meter(label: str, rendered: str) -> None: + match = COMPACTING_RE.search(rendered) + if match is None: + raise AssertionError( + f"{label}: server-reported usage did not trigger pre-send " + f"compaction:\n{rendered}" + ) + observed = int(match.group(1)) + # The provider sample is the lower bound. A tool result appended after that + # sample should increase the anchored effective meter slightly, while the + # compaction still runs before the advertised context wall. + if not MIDTURN_TOTAL_TOKENS <= observed < MIDTURN_CONTEXT_TOKENS: + raise AssertionError( + f"{label}: compacted at {observed} tokens, expected at least the " + f"{MIDTURN_TOTAL_TOKENS} provider sample and below the " + f"{MIDTURN_CONTEXT_TOKENS} context window" + ) def run_scenario( @@ -34,7 +206,7 @@ def run_scenario( expect_ws: int, expect_sse: int, health: str, -) -> None: +) -> int: env = { "HOME": tmp, "CODEX_HOME": codex_home, @@ -70,10 +242,15 @@ def run_scenario( session.wait_for_literal(REPLY_TEXT, start=cursor) session.wait_for(METER_RE, start=cursor) meter = METER_RE.search(terminal_text(bytes(session.raw[cursor:]))) - ctx_k, compact_k = int(meter.group(1)), int(meter.group(2)) - if compact_k != ctx_k * 8 // 10: + used = int(meter.group(1)) * (1000 if meter.group(2) == "k" else 1) + ctx_k, pct, compact_k = map(int, meter.group(3, 4, 5)) + if used <= 0 or not 0 <= pct <= 100: + raise AssertionError( + f"{label}: invalid context meter values used={used} pct={pct}" + ) + if ctx_k <= 0 or not 79 * ctx_k <= 100 * compact_k <= 81 * ctx_k: raise AssertionError( - f"{label}: meter compact@{compact_k}k is not 80% of {ctx_k}k ctx" + f"{label}: inconsistent meter context={ctx_k}k compact@={compact_k}k" ) if mock.ws_turns != expect_ws or mock.sse_turns != expect_sse: raise AssertionError( @@ -91,15 +268,329 @@ def run_scenario( f"{label}: REPL did not exit cleanly: " f"exit={result.exit_code} timed_out={result.timed_out}" ) + return ctx_k * 1000 + + +def assert_midturn_requests(mock: CodexMock) -> None: + requests = mock.recorded_requests() + if len(requests) != 3: + raise AssertionError( + f"midturn: expected exactly 3 model requests, got {len(requests)}: {requests!r}" + ) + first, compact, final = requests + transports = [request.transport for request in requests] + if transports != ["ws", "sse", "ws"]: + raise AssertionError(f"midturn: expected WS -> SSE -> WS, got {transports!r}") + if mock.ws_turns != 2 or mock.sse_turns != 1 or mock.ws_connections != 2: + raise AssertionError( + "midturn: expected two turns on two fresh WS connections plus one SSE " + f"turn; ws_turns={mock.ws_turns} sse_turns={mock.sse_turns} " + f"ws_connections={mock.ws_connections}" + ) + if first.connection_id == final.connection_id: + raise AssertionError("midturn: final request reused the pre-compaction WS") + for request in requests: + if "previous_response_id" in request.body: + raise AssertionError( + f"midturn: request {request.ordinal} carried stale previous_response_id: " + f"{request.body['previous_response_id']!r}" + ) + + first_input = first.body.get("input") + if ( + first.body.get("type") != "response.create" + or not isinstance(first_input, list) + or not any(user_text(item) == MIDTURN_PROMPT for item in first_input) + or "tools" not in first.body + ): + raise AssertionError( + f"midturn: first WS request was not full tool-enabled input: {first.body!r}" + ) + + compact_input = compact.body.get("input") + compact_texts = ( + [text for item in compact_input if (text := user_text(item)) is not None] + if isinstance(compact_input, list) + else [] + ) + compact_types = ( + [item.get("type") for item in compact_input if isinstance(item, dict)] + if isinstance(compact_input, list) + else [] + ) + compact_json = json.dumps(compact.body, separators=(",", ":")) + if "reasoning" in compact_types or MIDTURN_REASONING_MARKER in compact_json: + raise AssertionError( + "midturn: SSE compaction request retained the active-loop reasoning item" + ) + if ( + compact.connection_id is not None + or "tools" in compact.body + or MIDTURN_PROMPT not in compact_texts + or "function_call" not in compact_types + or "function_call_output" not in compact_types + or not any( + text.startswith("Summarize this entire conversation") + for text in compact_texts + ) + ): + raise AssertionError( + f"midturn: compaction request was not a full, tool-free SSE summary request: {compact.body!r}" + ) + + final_input = final.body.get("input") + final_texts = ( + [text for item in final_input if (text := user_text(item)) is not None] + if isinstance(final_input, list) + else [] + ) + if ( + final.body.get("type") != "response.create" + or not isinstance(final_input, list) + or len(final_input) != 1 + or len(final_texts) != 1 + or not final_texts[0].startswith( + "Context: the prior conversation was compacted" + ) + or MIDTURN_SUMMARY not in final_texts[0] + or "tools" not in final.body + ): + raise AssertionError( + "midturn: final WS request did not fully re-anchor on the handoff: " + f"{final.body!r}" + ) + + +def assert_transactional_requests(mock: CodexMock) -> None: + requests = mock.recorded_requests() + if len(requests) != 3: + raise AssertionError( + "transactional: expected exactly 3 model requests, " + f"got {len(requests)}: {requests!r}" + ) + first, compact, final = requests + transports = [request.transport for request in requests] + if transports != ["ws", "sse", "ws"]: + raise AssertionError( + f"transactional: expected WS -> SSE -> WS, got {transports!r}" + ) + if mock.ws_turns != 2 or mock.sse_turns != 1 or mock.ws_connections != 2: + raise AssertionError( + "transactional: expected two turns on two fresh WS connections plus " + f"one SSE turn; ws_turns={mock.ws_turns} sse_turns={mock.sse_turns} " + f"ws_connections={mock.ws_connections}" + ) + if first.connection_id == final.connection_id: + raise AssertionError( + "transactional: final request reused the pre-compaction WS" + ) + for request in requests: + if "previous_response_id" in request.body: + raise AssertionError( + f"transactional: request {request.ordinal} carried stale " + f"previous_response_id: {request.body['previous_response_id']!r}" + ) + + first_input = first.body.get("input") + if ( + first.body.get("type") != "response.create" + or not isinstance(first_input, list) + or not any(user_text(item) == TRANSACTIONAL_PROMPT for item in first_input) + or "tools" not in first.body + ): + raise AssertionError( + "transactional: first WS request was not full tool-enabled input: " + f"{first.body!r}" + ) + + compact_input = compact.body.get("input") + compact_texts = ( + [text for item in compact_input if (text := user_text(item)) is not None] + if isinstance(compact_input, list) + else [] + ) + compact_json = json.dumps(compact.body, separators=(",", ":")) + if ( + compact.connection_id is not None + or "tools" in compact.body + or TRANSACTIONAL_PROMPT not in compact_texts + or TRANSACTIONAL_REASONING_MARKER in compact_json + or not any( + text.startswith("Summarize this entire conversation") + for text in compact_texts + ) + ): + raise AssertionError( + "transactional: second request was not the pruned, synthetic SSE " + f"summary request: {compact.body!r}" + ) + + final_input = final.body.get("input") + final_texts = ( + [text for item in final_input if (text := user_text(item)) is not None] + if isinstance(final_input, list) + else [] + ) + final_objects = ( + [item for item in final_input if isinstance(item, dict)] + if isinstance(final_input, list) + else [] + ) + reasoning = [ + item + for item in final_objects + if item.get("type") == "reasoning" and item.get("id") == "rs_transactional_1" + ] + calls = [ + item + for item in final_objects + if item.get("type") == "function_call" + and item.get("id") == "fc_transactional_1" + and item.get("call_id") == TRANSACTIONAL_CALL_ID + ] + outputs = [ + item + for item in final_objects + if item.get("type") == "function_call_output" + and item.get("call_id") == TRANSACTIONAL_CALL_ID + and isinstance(item.get("output"), str) + and len(item["output"]) > 0 + ] + leaked_summary_response = any( + item.get("id") == "msg_transactional_empty_summary" for item in final_objects + ) + if ( + final.body.get("type") != "response.create" + or not isinstance(final_input, list) + or "tools" not in final.body + or TRANSACTIONAL_PROMPT not in final_texts + or any( + text.startswith("Summarize this entire conversation") + for text in final_texts + ) + or not any( + isinstance(item.get("encrypted_content"), str) + and item["encrypted_content"].startswith(TRANSACTIONAL_REASONING_MARKER) + and len(item["encrypted_content"]) == MIDTURN_REASONING_BYTES + for item in reasoning + ) + or len(calls) != 1 + or len(outputs) != 1 + or leaked_summary_response + ): + raise AssertionError( + "transactional: final WS request did not restore the original prompt, " + "reasoning, function call, and output without the synthetic compact " + f"instruction: {final.body!r}" + ) + + +def run_midturn_compaction_scenario( + tmp: str, codex_home: str, port: int, mock: CodexMock +) -> None: + env = { + "HOME": tmp, + "CODEX_HOME": codex_home, + "CODEGRAFF_API_KEY": "local-pty-test", + "GRAFF_FLEET": "off", + "GRAFF_NO_TELEMETRY": "1", + "GRAFF_CODEX_URL": f"http://127.0.0.1:{port}/backend-api/codex/responses", + } + ambient = tuple( + key + for key in os.environ + if (key.startswith("GRAFF_") or key.startswith("CODEX_") or key == "NO_COLOR") + and key not in env + ) + with PtySession( + GRAFF, + ["--model", "codex", "--no-telemetry"], + cwd=tmp, + env=env, + unset_env=ambient, + timeout=20.0, + ) as session: + session.wait_for_literal("] ›") + cursor = len(session.raw) + session.send_line(MIDTURN_PROMPT) + session.wait_for_literal(MIDTURN_FINAL, start=cursor) + session.wait_for_literal("history compacted to a", start=cursor) + session.pump_for(0.1) + rendered = terminal_text(bytes(session.raw[cursor:])) + assert_compaction_meter("midturn", rendered) + assert_midturn_requests(mock) + session.send_key("ctrl-d") + result = session.read_until_exit(5.0) + if result.timed_out or result.exit_code != 0: + raise SystemExit( + "midturn: REPL did not exit cleanly: " + f"exit={result.exit_code} timed_out={result.timed_out}" + ) + + +def run_transactional_compaction_scenario( + tmp: str, codex_home: str, port: int, mock: CodexMock +) -> None: + env = { + "HOME": tmp, + "CODEX_HOME": codex_home, + "CODEGRAFF_API_KEY": "local-pty-test", + "GRAFF_FLEET": "off", + "GRAFF_NO_TELEMETRY": "1", + "GRAFF_CODEX_URL": f"http://127.0.0.1:{port}/backend-api/codex/responses", + } + ambient = tuple( + key + for key in os.environ + if (key.startswith("GRAFF_") or key.startswith("CODEX_") or key == "NO_COLOR") + and key not in env + ) + with PtySession( + GRAFF, + ["--model", "codex", "--no-telemetry"], + cwd=tmp, + env=env, + unset_env=ambient, + timeout=20.0, + ) as session: + session.wait_for_literal("] ›") + cursor = len(session.raw) + session.send_line(TRANSACTIONAL_PROMPT) + session.wait_for_literal( + "compaction failed: empty summary, history unchanged", start=cursor + ) + session.wait_for_literal(TRANSACTIONAL_FINAL, start=cursor) + session.pump_for(0.1) + rendered = terminal_text(bytes(session.raw[cursor:])) + assert_compaction_meter("transactional", rendered) + if "history compacted to a" in rendered: + raise AssertionError( + f"transactional: empty summary was incorrectly installed:\n{rendered}" + ) + assert_transactional_requests(mock) + session.send_key("ctrl-d") + result = session.read_until_exit(5.0) + if result.timed_out or result.exit_code != 0: + raise SystemExit( + "transactional: REPL did not exit cleanly: " + f"exit={result.exit_code} timed_out={result.timed_out}" + ) def main() -> None: + global MIDTURN_CONTEXT_TOKENS, MIDTURN_TOTAL_TOKENS + with tempfile.TemporaryDirectory(prefix="graff-pty-codex-") as tmp: codex_home = os.path.join(tmp, "codex-home") os.makedirs(codex_home, exist_ok=True) with open(os.path.join(codex_home, "auth.json"), "w", encoding="utf-8") as fh: json.dump( - {"tokens": {"access_token": "pty-mock-token", "account_id": "acct-pty-mock"}}, + { + "tokens": { + "access_token": "pty-mock-token", + "account_id": "acct-pty-mock", + } + }, fh, ) @@ -109,7 +600,9 @@ def main() -> None: # the session cwd's .harness/settings.json. harness_dir = os.path.join(tmp, ".harness") os.makedirs(harness_dir, exist_ok=True) - with open(os.path.join(harness_dir, "settings.json"), "w", encoding="utf-8") as fh: + with open( + os.path.join(harness_dir, "settings.json"), "w", encoding="utf-8" + ) as fh: json.dump({"ai_title": False}, fh) scenarios = [ @@ -138,11 +631,12 @@ def main() -> None: "ok codex WS disabled: SSE reply + ctx meter + forced-off health", ), ] + runtime_context = None for label, extra_env, expect_ws, expect_sse, health, ok_line in scenarios: mock = CodexMock() port = mock.start() try: - run_scenario( + observed_context = run_scenario( label, tmp, codex_home, @@ -155,8 +649,41 @@ def main() -> None: ) finally: mock.stop() + if runtime_context is not None and observed_context != runtime_context: + raise AssertionError( + f"{label}: runtime context changed from {runtime_context} " + f"to {observed_context} across smoke scenarios" + ) + runtime_context = observed_context print(ok_line) + if runtime_context is None: + raise AssertionError("no smoke scenario reported a runtime context") + MIDTURN_CONTEXT_TOKENS = runtime_context + MIDTURN_TOTAL_TOKENS = runtime_context * 9 // 10 + + mock = CodexMock(events_for_request=midturn_events) + port = mock.start() + try: + run_midturn_compaction_scenario(tmp, codex_home, port, mock) + finally: + mock.stop() + print( + "ok codex server meter + reasoning prune: WS tool call -> " + "SSE compaction -> fresh full-input WS" + ) + + mock = CodexMock(events_for_request=transactional_events) + port = mock.start() + try: + run_transactional_compaction_scenario(tmp, codex_home, port, mock) + finally: + mock.stop() + print( + "ok empty mid-turn summary: transactional rollback -> " + "fresh full-input WS" + ) + if __name__ == "__main__": main() diff --git a/scripts/test-pty-repl.py b/scripts/test-pty-repl.py index 28cf5234..fedeaa53 100755 --- a/scripts/test-pty-repl.py +++ b/scripts/test-pty-repl.py @@ -46,16 +46,18 @@ def command( raise AssertionError(f"missing colored badge {color_bytes!r} after {line}") base = "deepseek-v4-pro · Extra high · codegraff" + # Match through cwd but not the closing bracket: the live context + # meter is appended after cwd and changes as the session evolves. command( "/effort xhigh", "reasoning effort: Extra high", - f"[{base} · cwd {tmp}]", + f"[{base} · cwd {tmp}", b"\x1b[35mExtra high\x1b[0m", ) command( "/model definitely-not-a-model", "unknown model 'definitely-not-a-model'", - f"[{base} · cwd {tmp}]", + f"[{base} · cwd {tmp}", None, ) cursor = len(session.raw) @@ -63,7 +65,7 @@ def command( session.wait_for_literal("/models [health]", start=cursor) session.wait_for_literal("/fallback [allow|remove|off]", start=cursor) session.wait_for_literal("/login [codegraff|codex|kimi]", start=cursor) - session.wait_for_literal(f"[{base} · cwd {tmp}]", start=cursor) + session.wait_for_literal(f"[{base} · cwd {tmp}", start=cursor) cursor = len(session.raw) session.send_line("/models health") session.wait_for_literal("active: deepseek-v4-pro via codegraff", start=cursor) @@ -74,31 +76,31 @@ def command( ) session.wait_for(r"✓\s+codegraff\s+environment", start=cursor) session.wait_for(r"·\s+codex\s+missing", start=cursor) - session.wait_for_literal(f"[{base} · cwd {tmp}]", start=cursor) + session.wait_for_literal(f"[{base} · cwd {tmp}", start=cursor) if b"local-pty-test" in bytes(session.raw[cursor:]): raise AssertionError("/models health exposed a credential value") command( "/plan", "plan mode on", - f"[{base} · Plan · cwd {tmp}]", + f"[{base} · Plan · cwd {tmp}", b"\x1b[33mPlan\x1b[0m", ) command( "/strict", "strict mode ON", - f"[{base} · Plan · Strict · cwd {tmp}]", + f"[{base} · Plan · Strict · cwd {tmp}", b"\x1b[31mStrict\x1b[0m", ) command( "/ultracode on", "ultracode mode: on", - f"[{base} · Plan · Strict · Ultracode · cwd {tmp}]", + f"[{base} · Plan · Strict · Ultracode · cwd {tmp}", b"\x1b[35mUltracode\x1b[0m", ) command( "/yolo", "yolo mode ON", - f"[{base} · Plan · Strict · Ultracode · cwd {tmp}]", + f"[{base} · Plan · Strict · Ultracode · cwd {tmp}", None, ) if b"\x1b[31mYOLO\x1b[0m" in bytes(session.raw): @@ -106,7 +108,7 @@ def command( cursor = len(session.raw) session.send_line("/key no-such-provider supersecret") session.wait_for_literal("unknown provider 'no-such-provider'", start=cursor) - session.wait_for_literal(f"cwd {tmp}]", start=cursor) + session.wait_for_literal(f"cwd {tmp}", start=cursor) if b"supersecret" in bytes(session.raw[cursor:]): raise AssertionError("/key secret was echoed to the terminal") session.send_key("ctrl-d") diff --git a/scripts/test-title-parallel.py b/scripts/test-title-parallel.py new file mode 100644 index 00000000..6306c4cd --- /dev/null +++ b/scripts/test-title-parallel.py @@ -0,0 +1,148 @@ +#!/usr/bin/env python3 +"""Prove the first-turn AI title request overlaps the real model turn.""" + +import json +import os +import sys +import tempfile +import threading +import time + +from codex_ws_mock import CodexMock +from pty_harness import PtySession + + +_arg = sys.argv[1] if len(sys.argv) > 1 else "graff" +GRAFF = os.path.abspath(_arg) if os.sep in _arg else _arg +MAIN_REPLY = "PARALLEL_TITLE_MAIN_OK" + + +def message_item(text: str, item_id: str) -> dict: + return { + "type": "message", + "id": item_id, + "status": "completed", + "role": "assistant", + "content": [ + {"type": "output_text", "text": text, "annotations": []} + ], + } + + +def main() -> None: + barrier = threading.Barrier(2) + lock = threading.Lock() + arrivals: list[tuple[float, str]] = [] + + def events(request) -> list[dict]: + instructions = request.body.get("instructions", "") + is_title = "You summarize what a coding session is about" in instructions + with lock: + arrivals.append((time.monotonic(), "title" if is_title else "main")) + # A serialized implementation blocks here for three seconds before the + # main request can start; true overlap sends both through together. + try: + barrier.wait(timeout=3.0) + except threading.BrokenBarrierError: + pass + text = "Parallel title generation" if is_title else MAIN_REPLY + return [ + { + "type": "response.output_item.done", + "item": message_item(text, f"msg_parallel_{request.ordinal}"), + }, + { + "type": "response.completed", + "response": { + "id": f"resp_parallel_{request.ordinal}", + "usage": { + "input_tokens": 100, + "input_tokens_details": {"cached_tokens": 0}, + "output_tokens": 10, + "total_tokens": 110, + }, + }, + }, + ] + + mock = CodexMock(events_for_request=events) + port = mock.start() + try: + with tempfile.TemporaryDirectory(prefix="graff-title-parallel-") as tmp: + codex_home = os.path.join(tmp, "codex-home") + os.makedirs(codex_home) + with open( + os.path.join(codex_home, "auth.json"), "w", encoding="utf-8" + ) as fh: + json.dump( + { + "tokens": { + "access_token": "title-parallel-mock", + "account_id": "acct-title-parallel", + } + }, + fh, + ) + harness_dir = os.path.join(tmp, ".harness") + os.makedirs(harness_dir) + with open( + os.path.join(harness_dir, "settings.json"), + "w", + encoding="utf-8", + ) as fh: + json.dump({"skills": {"codedbpro": False}}, fh) + + env = { + "HOME": tmp, + "CODEX_HOME": codex_home, + "GRAFF_CODEX_URL": ( + f"http://127.0.0.1:{port}/backend-api/codex/responses" + ), + "GRAFF_CODEX_WS": "off", + "GRAFF_FLEET": "off", + "GRAFF_NO_TELEMETRY": "1", + } + ambient = tuple( + name + for name in os.environ + if ( + name.startswith("GRAFF_") + or name.startswith("CODEX_") + or name == "NO_COLOR" + ) + and name not in env + ) + with PtySession( + GRAFF, + ["--model", "codex", "--no-telemetry"], + cwd=tmp, + env=env, + unset_env=ambient, + timeout=10.0, + ) as session: + session.wait_for_literal("] ›") + cursor = len(session.raw) + session.send_line("make the title and answer overlap") + session.wait_for_literal(MAIN_REPLY, start=cursor) + session.wait_for_literal("] ›", start=cursor) + session.send_key("ctrl-d") + result = session.read_until_exit(5.0) + if result.timed_out or result.exit_code != 0: + raise AssertionError( + f"session exit={result.exit_code} timed_out={result.timed_out}" + ) + finally: + mock.stop() + + with lock: + observed = sorted(arrivals) + if len(observed) != 2 or {kind for _, kind in observed} != {"title", "main"}: + raise AssertionError(f"expected one title and one main request: {observed!r}") + delta_ms = (observed[1][0] - observed[0][0]) * 1000 + if delta_ms > 750: + raise AssertionError(f"title/main requests serialized ({delta_ms:.1f}ms apart)") + print(f"ok AI title and main turn overlapped ({delta_ms:.1f}ms apart)") + + +if __name__ == "__main__": + main() diff --git a/sdk/py/harness_sdk.py b/sdk/py/harness_sdk.py index 06ddb9c6..95ced6c2 100644 --- a/sdk/py/harness_sdk.py +++ b/sdk/py/harness_sdk.py @@ -14,7 +14,7 @@ from typing import Iterator, Optional MODELS = ["MiniMax-M2.5", "MiniMax-M2.7", "MiniMax-M3", "accounts/fireworks/models/deepseek-v4-flash", "accounts/fireworks/models/deepseek-v4-pro", "accounts/fireworks/models/glm-5p2", "accounts/fireworks/models/gpt-oss-120b", "accounts/fireworks/models/kimi-k2p6", "accounts/fireworks/models/kimi-k2p7-code", "accounts/fireworks/models/minimax-m3", "accounts/fireworks/models/qwen3p7-plus", "claude-fable-5", "claude-haiku-4-5", "claude-opus-4-5", "claude-opus-4-6", "claude-opus-4-7", "claude-opus-4-8", "claude-opus-4.8", "claude-sonnet-4-5", "claude-sonnet-4-6", "claude-sonnet-4.6", "deepseek-chat", "deepseek-reasoner", "deepseek-v4-flash", "deepseek-v4-pro", "fugu", "fugu-ultra", "fugu-ultra-20260615", "glm-4.5", "glm-4.7", "glm-5", "glm-5.2", "gpt-5-codex", "gpt-5.2", "gpt-5.3-codex-spark", "gpt-5.4", "gpt-5.4-mini", "gpt-5.4-pro", "gpt-5.5", "gpt-5.6", "gpt-5.6-luna", "gpt-5.6-sol", "gpt-5.6-terra", "grok-4.3", "grok-build", "kimi-k2.6", "kimi-k2.7", "kimi-latest", "lmstudio", "mimo-v2-flash", "mimo-v2.5", "mimo-v2.5-pro", "mimo-v2.5-pro-ultraspeed", "minimax-m3", "mlx-community/Qwen3.6-27B-OptiQ-4bit"] -TOOLS = ["bash", "bash_output", "bash_kill", "read_file", "edit_file", "write_file", "webfetch", "codedb", "todo_write", "todo_read", "eval", "ask_user", "attempt_completion", "subagent", "workflow"] +TOOLS = ["bash", "bash_output", "bash_kill", "read_file", "edit_file", "write_file", "webfetch", "codedb", "todo_write", "todo_read", "eval", "ask_user", "attempt_completion", "clock_sleep", "subagent", "workflow"] PROVIDERS = ["anthropic", "codegraff", "deepseek", "openai", "minimax", "xiaomi", "kimi", "moonshot", "xai", "zai", "fugu", "fireworks", "mlx", "lmstudio", "codex"] diff --git a/sdk/ts/harness.ts b/sdk/ts/harness.ts index 5c0cfda7..633b5638 100644 --- a/sdk/ts/harness.ts +++ b/sdk/ts/harness.ts @@ -11,7 +11,7 @@ import { createInterface } from "node:readline"; export const HARNESS_VERSION = "0.4"; export type ModelName = "MiniMax-M2.5" | "MiniMax-M2.7" | "MiniMax-M3" | "accounts/fireworks/models/deepseek-v4-flash" | "accounts/fireworks/models/deepseek-v4-pro" | "accounts/fireworks/models/glm-5p2" | "accounts/fireworks/models/gpt-oss-120b" | "accounts/fireworks/models/kimi-k2p6" | "accounts/fireworks/models/kimi-k2p7-code" | "accounts/fireworks/models/minimax-m3" | "accounts/fireworks/models/qwen3p7-plus" | "claude-fable-5" | "claude-haiku-4-5" | "claude-opus-4-5" | "claude-opus-4-6" | "claude-opus-4-7" | "claude-opus-4-8" | "claude-opus-4.8" | "claude-sonnet-4-5" | "claude-sonnet-4-6" | "claude-sonnet-4.6" | "deepseek-chat" | "deepseek-reasoner" | "deepseek-v4-flash" | "deepseek-v4-pro" | "fugu" | "fugu-ultra" | "fugu-ultra-20260615" | "glm-4.5" | "glm-4.7" | "glm-5" | "glm-5.2" | "gpt-5-codex" | "gpt-5.2" | "gpt-5.3-codex-spark" | "gpt-5.4" | "gpt-5.4-mini" | "gpt-5.4-pro" | "gpt-5.5" | "gpt-5.6" | "gpt-5.6-luna" | "gpt-5.6-sol" | "gpt-5.6-terra" | "grok-4.3" | "grok-build" | "kimi-k2.6" | "kimi-k2.7" | "kimi-latest" | "lmstudio" | "mimo-v2-flash" | "mimo-v2.5" | "mimo-v2.5-pro" | "mimo-v2.5-pro-ultraspeed" | "minimax-m3" | "mlx-community/Qwen3.6-27B-OptiQ-4bit" | (string & {}); -export type ToolName = "bash" | "bash_output" | "bash_kill" | "read_file" | "edit_file" | "write_file" | "webfetch" | "codedb" | "todo_write" | "todo_read" | "eval" | "ask_user" | "attempt_completion" | "subagent" | "workflow"; +export type ToolName = "bash" | "bash_output" | "bash_kill" | "read_file" | "edit_file" | "write_file" | "webfetch" | "codedb" | "todo_write" | "todo_read" | "eval" | "ask_user" | "attempt_completion" | "clock_sleep" | "subagent" | "workflow"; export type ProviderId = "anthropic" | "codegraff" | "deepseek" | "openai" | "minimax" | "xiaomi" | "kimi" | "moonshot" | "xai" | "zai" | "fugu" | "fireworks" | "mlx" | "lmstudio" | "codex"; /** Events streamed by `harness --json` (one JSON object per stdout line), each @@ -417,4 +417,4 @@ export async function* runAgent(opts: RunAgentOptions): AsyncGenerator { } export const MODELS: ModelName[] = ["MiniMax-M2.5", "MiniMax-M2.7", "MiniMax-M3", "accounts/fireworks/models/deepseek-v4-flash", "accounts/fireworks/models/deepseek-v4-pro", "accounts/fireworks/models/glm-5p2", "accounts/fireworks/models/gpt-oss-120b", "accounts/fireworks/models/kimi-k2p6", "accounts/fireworks/models/kimi-k2p7-code", "accounts/fireworks/models/minimax-m3", "accounts/fireworks/models/qwen3p7-plus", "claude-fable-5", "claude-haiku-4-5", "claude-opus-4-5", "claude-opus-4-6", "claude-opus-4-7", "claude-opus-4-8", "claude-opus-4.8", "claude-sonnet-4-5", "claude-sonnet-4-6", "claude-sonnet-4.6", "deepseek-chat", "deepseek-reasoner", "deepseek-v4-flash", "deepseek-v4-pro", "fugu", "fugu-ultra", "fugu-ultra-20260615", "glm-4.5", "glm-4.7", "glm-5", "glm-5.2", "gpt-5-codex", "gpt-5.2", "gpt-5.3-codex-spark", "gpt-5.4", "gpt-5.4-mini", "gpt-5.4-pro", "gpt-5.5", "gpt-5.6", "gpt-5.6-luna", "gpt-5.6-sol", "gpt-5.6-terra", "grok-4.3", "grok-build", "kimi-k2.6", "kimi-k2.7", "kimi-latest", "lmstudio", "mimo-v2-flash", "mimo-v2.5", "mimo-v2.5-pro", "mimo-v2.5-pro-ultraspeed", "minimax-m3", "mlx-community/Qwen3.6-27B-OptiQ-4bit"]; -export const TOOLS: ToolName[] = ["bash", "bash_output", "bash_kill", "read_file", "edit_file", "write_file", "webfetch", "codedb", "todo_write", "todo_read", "eval", "ask_user", "attempt_completion", "subagent", "workflow"]; +export const TOOLS: ToolName[] = ["bash", "bash_output", "bash_kill", "read_file", "edit_file", "write_file", "webfetch", "codedb", "todo_write", "todo_read", "eval", "ask_user", "attempt_completion", "clock_sleep", "subagent", "workflow"]; diff --git a/sdk/ts/remote.ts b/sdk/ts/remote.ts index ced29c86..5964f7d5 100644 --- a/sdk/ts/remote.ts +++ b/sdk/ts/remote.ts @@ -10,7 +10,7 @@ export const HARNESS_VERSION = "0.4"; export type ModelName = "MiniMax-M2.5" | "MiniMax-M2.7" | "MiniMax-M3" | "accounts/fireworks/models/deepseek-v4-flash" | "accounts/fireworks/models/deepseek-v4-pro" | "accounts/fireworks/models/glm-5p2" | "accounts/fireworks/models/gpt-oss-120b" | "accounts/fireworks/models/kimi-k2p6" | "accounts/fireworks/models/kimi-k2p7-code" | "accounts/fireworks/models/minimax-m3" | "accounts/fireworks/models/qwen3p7-plus" | "claude-fable-5" | "claude-haiku-4-5" | "claude-opus-4-5" | "claude-opus-4-6" | "claude-opus-4-7" | "claude-opus-4-8" | "claude-opus-4.8" | "claude-sonnet-4-5" | "claude-sonnet-4-6" | "claude-sonnet-4.6" | "deepseek-chat" | "deepseek-reasoner" | "deepseek-v4-flash" | "deepseek-v4-pro" | "fugu" | "fugu-ultra" | "fugu-ultra-20260615" | "glm-4.5" | "glm-4.7" | "glm-5" | "glm-5.2" | "gpt-5-codex" | "gpt-5.2" | "gpt-5.3-codex-spark" | "gpt-5.4" | "gpt-5.4-mini" | "gpt-5.4-pro" | "gpt-5.5" | "gpt-5.6" | "gpt-5.6-luna" | "gpt-5.6-sol" | "gpt-5.6-terra" | "grok-4.3" | "grok-build" | "kimi-k2.6" | "kimi-k2.7" | "kimi-latest" | "lmstudio" | "mimo-v2-flash" | "mimo-v2.5" | "mimo-v2.5-pro" | "mimo-v2.5-pro-ultraspeed" | "minimax-m3" | "mlx-community/Qwen3.6-27B-OptiQ-4bit" | (string & {}); -export type ToolName = "bash" | "bash_output" | "bash_kill" | "read_file" | "edit_file" | "write_file" | "webfetch" | "codedb" | "todo_write" | "todo_read" | "eval" | "ask_user" | "attempt_completion" | "subagent" | "workflow"; +export type ToolName = "bash" | "bash_output" | "bash_kill" | "read_file" | "edit_file" | "write_file" | "webfetch" | "codedb" | "todo_write" | "todo_read" | "eval" | "ask_user" | "attempt_completion" | "clock_sleep" | "subagent" | "workflow"; export type ProviderId = "anthropic" | "codegraff" | "deepseek" | "openai" | "minimax" | "xiaomi" | "kimi" | "moonshot" | "xai" | "zai" | "fugu" | "fireworks" | "mlx" | "lmstudio" | "codex"; /** Events streamed by the bridge (same `--json` contract as the stdio SDK). @@ -247,4 +247,4 @@ export async function* runAgentRemote(opts: RunAgentRemoteOptions): AsyncGenerat } export const MODELS: ModelName[] = ["MiniMax-M2.5", "MiniMax-M2.7", "MiniMax-M3", "accounts/fireworks/models/deepseek-v4-flash", "accounts/fireworks/models/deepseek-v4-pro", "accounts/fireworks/models/glm-5p2", "accounts/fireworks/models/gpt-oss-120b", "accounts/fireworks/models/kimi-k2p6", "accounts/fireworks/models/kimi-k2p7-code", "accounts/fireworks/models/minimax-m3", "accounts/fireworks/models/qwen3p7-plus", "claude-fable-5", "claude-haiku-4-5", "claude-opus-4-5", "claude-opus-4-6", "claude-opus-4-7", "claude-opus-4-8", "claude-opus-4.8", "claude-sonnet-4-5", "claude-sonnet-4-6", "claude-sonnet-4.6", "deepseek-chat", "deepseek-reasoner", "deepseek-v4-flash", "deepseek-v4-pro", "fugu", "fugu-ultra", "fugu-ultra-20260615", "glm-4.5", "glm-4.7", "glm-5", "glm-5.2", "gpt-5-codex", "gpt-5.2", "gpt-5.3-codex-spark", "gpt-5.4", "gpt-5.4-mini", "gpt-5.4-pro", "gpt-5.5", "gpt-5.6", "gpt-5.6-luna", "gpt-5.6-sol", "gpt-5.6-terra", "grok-4.3", "grok-build", "kimi-k2.6", "kimi-k2.7", "kimi-latest", "lmstudio", "mimo-v2-flash", "mimo-v2.5", "mimo-v2.5-pro", "mimo-v2.5-pro-ultraspeed", "minimax-m3", "mlx-community/Qwen3.6-27B-OptiQ-4bit"]; -export const TOOLS: ToolName[] = ["bash", "bash_output", "bash_kill", "read_file", "edit_file", "write_file", "webfetch", "codedb", "todo_write", "todo_read", "eval", "ask_user", "attempt_completion", "subagent", "workflow"]; +export const TOOLS: ToolName[] = ["bash", "bash_output", "bash_kill", "read_file", "edit_file", "write_file", "webfetch", "codedb", "todo_write", "todo_read", "eval", "ask_user", "attempt_completion", "clock_sleep", "subagent", "workflow"]; diff --git a/social/graff-launch-01-size.png b/social/graff-launch-01-size.png new file mode 100644 index 00000000..f9f74c06 Binary files /dev/null and b/social/graff-launch-01-size.png differ diff --git a/social/graff-launch-02-dogfood.png b/social/graff-launch-02-dogfood.png new file mode 100644 index 00000000..a1455cad Binary files /dev/null and b/social/graff-launch-02-dogfood.png differ diff --git a/social/graff-launch-02-learning.png b/social/graff-launch-02-learning.png new file mode 100644 index 00000000..72bd320a Binary files /dev/null and b/social/graff-launch-02-learning.png differ diff --git a/social/graff-launch-03-models-oauth.png b/social/graff-launch-03-models-oauth.png new file mode 100644 index 00000000..04b8ee3e Binary files /dev/null and b/social/graff-launch-03-models-oauth.png differ diff --git a/social/graff-launch-04-fanout-worktree-sdk.png b/social/graff-launch-04-fanout-worktree-sdk.png new file mode 100644 index 00000000..3849785a Binary files /dev/null and b/social/graff-launch-04-fanout-worktree-sdk.png differ diff --git a/social/graff-launch-06-oauth.png b/social/graff-launch-06-oauth.png new file mode 100644 index 00000000..c57344b6 Binary files /dev/null and b/social/graff-launch-06-oauth.png differ diff --git a/social/graff-launch-07-sdks.png b/social/graff-launch-07-sdks.png new file mode 100644 index 00000000..b9ee7801 Binary files /dev/null and b/social/graff-launch-07-sdks.png differ diff --git a/social/graff-launch-08-surfaces-cta.png b/social/graff-launch-08-surfaces-cta.png new file mode 100644 index 00000000..005c7f31 Binary files /dev/null and b/social/graff-launch-08-surfaces-cta.png differ diff --git a/social/graff-launch-09-credits.png b/social/graff-launch-09-credits.png new file mode 100644 index 00000000..b18b56bf Binary files /dev/null and b/social/graff-launch-09-credits.png differ diff --git a/socials/release-post/linkedin.md b/socials/release-post/linkedin.md new file mode 100644 index 00000000..746678f0 --- /dev/null +++ b/socials/release-post/linkedin.md @@ -0,0 +1,43 @@ +# graff launch post (LinkedIn) + +A single-post LinkedIn version of the graff launch, mirroring the X thread. LinkedIn renders no markdown, so the post text below is plain (no bold, no backticks). First line is the hook that shows before the "...more" fold. Every claim verified against source; same honesty guardrails as the thread. + +--- + +Most coding agents are a few hundred megabytes of runtime that forget everything the moment you close them. We built the opposite. + +Today we are launching graff: a continual learning coding harness that ships as a single native binary a couple of megabytes in size, starts in milliseconds, and runs in tens of MB of RAM. + +Here is what it does: + +→ Runs any model. 15 providers are built in (Anthropic, OpenAI, DeepSeek, xAI Grok, Kimi, MiniMax, GLM, Sakana Fugu, Fireworks and more), plus local models via MLX on Apple Silicon and LM Studio. One flag swaps the model, and /fallback hops to another provider when one goes down. + +→ Logs in, no key-hunting. "graff login codex" signs you into ChatGPT and runs Codex over OpenAI's Responses API. "graff login xai" runs real OAuth into your Grok account, with tokens that refresh themselves. + +→ Fans out. It spawns sandboxed subagents that run in parallel and report back, composed by a workflow tool into phased or pipelined fleets, with git worktree isolation for risky work. + +→ Drives from your own code. Official SDKs on PyPI (pip install codegraff) and npm (npm i @codegraff/sdk), with the same typed API in both. + +The part we are most excited about is the continual learning. + +graff keeps an archive of every run and scores how it went, then keeps the best version of its own prompt for each kind of work: reviewing, researching, implementing, staying skeptical. Run /agents promote and it gets better the more you use it. Fleet-wide learning across everyone's runs, sharing only the prompt and a signed score and never your code, is landing next. + +It has been a collaboration from day one, built with new contributors like yxlyx, and it stands on the shoulders of giants: the MAP-Elites and Darwin-Gödel-Machine research from Jeff Clune and others. + +Built in Zig. Available as a CLI, a desktop app, an iOS companion, and a built-in REPL. + +Try it: github.com/justrach/codegraff + +What would you want a coding agent to learn first? + +#AI #DeveloperTools #OpenSource #CodingAgents #Zig #ContinualLearning + +--- + +## Posting notes + +- The hook (first line) is ~130 chars so it lands fully before LinkedIn's "...more" fold on mobile. +- Plain text on purpose: LinkedIn renders no markdown, so no bold/backticks. Commands are set off with quotes; arrows (→) and hashtags are literal unicode that LinkedIn shows fine. +- To actually tag people, retype the @ mentions in the LinkedIn composer and pick their profiles (yxlyx, Jeff Clune). Plain names here will not auto-link. +- Same honesty guardrails as the thread: the continual-learning claim is the shipped local loop (archive + scored fitness + /agents promote). It does NOT claim an evolved agent beat a baseline on held-out benchmarks (no such result in the repo). Fleet-wide learning is framed as "landing next," not live. Grok is real OAuth login only, not keyless inference (xAI still account-gates api.x.ai). Size is the ReleaseFast/shipped build (~3 MB), not the stale 1.7 MB figure. +- Swap in a nicer install one-liner/site if you have one, and trim hashtags to taste (3 to 5 perform best on LinkedIn). diff --git a/socials/release-post/linkedin2.md b/socials/release-post/linkedin2.md new file mode 100644 index 00000000..3631fa37 --- /dev/null +++ b/socials/release-post/linkedin2.md @@ -0,0 +1,61 @@ +# graff launch post (LinkedIn) + +Final coworker framing for LinkedIn. Continual learning + tiny footprint lead. Plain text only. Same honesty guardrails as linkedin.md / twitter.md. + +--- + +Most AI coding tools feel like rented contractors. Heavy, forgetful, and gone the moment the session ends. They do not get better the more you work with them. + +My friend Yu Xi Lim and I wanted a different kind of teammate. + +What if you had a coding coworker who learned from every project, stayed light enough to keep on your machine, and got better the more people worked with them? + +That coworker is Graff. + +Today we are introducing Graff: a continual learning coding coworker. It remembers how each run went, scores what worked, and keeps getting sharper at reviewing, researching, implementing, and staying skeptical. The more you work with it, the better it gets. Fleet-wide learning across teams is landing next, so the more people who hire graff onto their stack, the better the whole crew gets. + +And this coworker is absurdly light. About 3 MB on disk. About 25 MB of RAM when focused. Spin up eight parallel teammates (with ultracode) and you are still around 15 MB total. Starts in milliseconds. Lightweight, powerful, and fast. Built in Zig. + +A few providers actually let you bring a subscription instead of buying API credits. ChatGPT, xAI / X Premium, and Kimi Code are the ones that do: "graff login codex", "graff login xai", or "graff login kimi". Everyone else still works (Anthropic, OpenAI API, DeepSeek, MiniMax, GLM, Sakana Fugu, local MLX / LM Studio, and more), but those run on normal API fees (or if they have coding access), or free if you run local. + +What else can this coworker do: + +• Fan work out to sandboxed teammates in parallel, with git worktree isolation for risky changes +• Plug into your own systems via SDKs on PyPI and npm (https://lnkd.in/gNGxyCes) + +The goal is not another confident chatbot on your team. It is a coworker that stays small, stays yours, and compounds with experience. + +And Graff is already on the job. It is the coding harness behind Lawplain, and a few startups are using it too. + +Open source at https://lnkd.in/gegjqJpb, with docs and the rest of the suite at https://codegraff.com. + +We are building an ecosystem around this teammate, not a one-off demo. + +Longer walkthrough from my AIE talk: https://lnkd.in/gYwbWBid + +This is only day one. More is coming, including automated coding reviews where you pick the model (DeepSeek V4 Pro and friends). + +A full PR review can already land around $0.41: https://lnkd.in/gZJizN6E + +It stands on the shoulders of giants: the MAP-Elites and Darwin-Gödel-Machine research from Jeff Clune, Cong Lu, Shengran Hu, and others. Huge thanks to Pranav Pappu, Gabriel Chua, and Stefania Druga for contributing along the way. + +Bring Graff onto your team, tell us what breaks, and reach out to Yu Xi or me. + +curl -fsSL https://lnkd.in/gSkDvbRh | bash + +https://codegraff.com + +#AI #DeveloperTools #OpenSource #CodingAgents #ContinualLearning + +--- + +## Posting notes + +- Framing: Graff as coworker/teammate for LinkedIn; "coding harness" kept once for Lawplain accuracy. +- Flow: rented contractor hook → Yu Xi Lim question → coworker + continual learning → tiny footprint / ultracode team → subscription logins → capabilities → proof/ecosystem/AIE → reviews → credits → hire CTA + install. +- Honesty: local continual learning ships (archive + score + /agents promote). Fleet-wide = landing next. Footprint: ~3 MB disk, ~25 MB focused RAM, ~15 MB for 8 parallel subagents (ultracode = multi-agent workflow mode). +- Subscription OAuth that actually exist in src/cli.zig / src/oauth.zig: graff login codex, graff login xai, graff login kimi (plus bare graff login for a codegraff key). Sakana Fugu is an API-key provider (FUGU_API_KEY), not a login flow — keep it in the "everyone else" list, not the subscription sentence. +- Cost example: ~$0.41 from merjs#100 ($0.409 billed; fusion → mimo-v2.5-pro). DeepSeek V4 Pro = selectable option. +- Install one-liner short link should resolve to the harness installer (codegraff.com/install-graff.sh), not the companion tools installer (codegraff.com/install.sh). +- Credits: Yu Xi Lim co-builder; Jeff Clune / Cong Lu / Shengran Hu for MAP-Elites / DGM lineage; Pranav Pappu, Gabriel Chua, Stefania Druga thanked. Tag people in the LinkedIn composer (names here will not auto-link). +- Credit commas smoothed for LinkedIn readability. diff --git a/socials/release-post/twitter.md b/socials/release-post/twitter.md new file mode 100644 index 00000000..58f63ec6 --- /dev/null +++ b/socials/release-post/twitter.md @@ -0,0 +1,122 @@ +# graff launch thread (coworker framing) + +A nine-post X thread mirroring the LinkedIn launch: Graff as a continual learning coding coworker. Same story beats as socials/release-post/linkedin2.md, cut for X. + +Every tweet is 280 characters or fewer. Tweet 1 has no link. Tweet copy contains no em dashes or backticks. + +Narrative arc: rented contractors → Yu Xi / different teammate → Graff + continual learning → light footprint / ultracode → subscription logins → fan-out + SDKs → already on the job → CTA → credits. + +--- + +**Tweet 1: hook (167 chars)** + +Most AI coding tools feel like rented contractors. Heavy, forgetful, and gone the moment the session ends. + +They do not get better the more you work with them. 🧵 (1/9) + +--- + +**Tweet 2: the question (223 chars)** + +2/ My friend @yxlyx and I wanted a different kind of teammate. + +What if you had a coding coworker who learned from every project, stayed light enough to keep on your machine, and got better the more people worked with them? + +--- + +**Tweet 3: introduce Graff (207 chars)** + +3/ That coworker is Graff. + +A continual learning coding coworker. It remembers how each run went, scores what worked, and keeps getting sharper at reviewing, researching, implementing, and staying skeptical. + +--- + +**Tweet 4: compounds + footprint (241 chars)** + +4/ The more you work with it, the better it gets. Fleet-wide learning across teams is landing next. + +And this coworker is absurdly light: about 3 MB on disk, about 25 MB of RAM when focused. Eight ultracode teammates still land around 15 MB. + +--- + +**Tweet 5: subscription logins (215 chars)** + +5/ Bring a subscription instead of buying API credits. + +graff login codex for ChatGPT. graff login xai for X Premium. graff login kimi for Kimi Code. + +Everyone else still works on API fees, or free if you run local. + +--- + +**Tweet 6: fan-out and SDKs (203 chars)** + +6/ What else can this coworker do? + +Fan work out to sandboxed teammates in parallel, with git worktree isolation for risky changes. + +Drive it from your code: pip install codegraff or npm i @codegraff/sdk + +--- + +**Tweet 7: already on the job (189 chars)** + +7/ Not another confident chatbot. A coworker that stays small, stays yours, and compounds with experience. + +Graff is already on the job behind Lawplain, and a few startups are using it too. + +--- + +**Tweet 8: close and CTA (164 raw chars, about 143 X-weighted)** + +8/ Open source. Built in Zig. Docs and the rest of the suite at codegraff.com + +github.com/justrach/codegraff + +curl -fsSL https://codegraff.com/install-graff.sh | sh + +--- + +**Tweet 9: credits (227 chars)** + +9/ Built with @yxlyx and contributors. Thanks to Pranav Pappu, Gabriel Chua, and Stefania Druga. + +Learning loop draws on MAP-Elites and Darwin Gödel Machine research by @jeffclune and collaborators. + +Bring Graff onto your team. + +--- + +## Card attachments + +- Tweet 3: /social/graff-launch-02-learning.png +- Tweet 4: /social/graff-launch-01-size.png +- Tweet 5: /social/graff-launch-03-models-oauth.png +- Tweet 6: /social/graff-launch-04-fanout-worktree-sdk.png +- Tweet 7: /social/graff-launch-02-dogfood.png +- Tweet 8: /social/graff-launch-08-surfaces-cta.png +- Tweet 9: /social/graff-launch-09-credits.png + +Optional extras if you want denser media: Tweet 5 can use /social/graff-launch-06-oauth.png instead of 03, and Tweet 6 can swap in /social/graff-launch-07-sdks.png. Prefer one card per tweet. + +## Posting notes + +- Structure mirrors linkedin2.md: rented contractors (1) -> @yxlyx teammate question (2) -> Graff coworker + learning (3) -> fleet next + footprint / ultracode (4) -> subscription logins (5) -> fan-out + SDKs (6) -> Lawplain proof (7) -> CTA / install (8) -> credits (9). +- Framing: coworker/teammate, not harness-first. Continual learning still leads the product story. +- Honesty: local continual learning ships (archive + score + /agents promote). Fleet-wide = landing next. +- Footprint: ~3 MB disk, ~25 MB focused RAM, ~15 MB for eight ultracode / parallel teammates. ultracode = multi-agent workflow mode. +- Subscription OAuth only: graff login codex, graff login xai, graff login kimi. Sakana Fugu and the rest are API-key / local, not login flows. +- Install one-liner uses install-graff.sh (harness), not install.sh (companion tools). +- Credits match LinkedIn: @yxlyx, Pranav Pappu, Gabriel Chua, Stefania Druga, @jeffclune / MAP-Elites / Darwin Gödel Machine. Tag handles in the X composer where available. + +## Source verification map + +- Binary and memory methodology: architecture.md and benchmarks/README.md +- ultracode multi-agent mode: src/mainloop.zig, README.md +- Evolutionary / lineage archive: README.md (An evolutionary harness), src/trace.zig +- Local promotion semantics: src/fleet.zig (promoteAgents) and src/commands_session.zig (/agents promote) +- Subscription OAuth: src/oauth.zig, src/cli.zig (codex, xai, kimi) +- Subagents, workflows, worktree: src/subagent.zig, src/schema.zig, src/workflow.zig, src/session_start.zig +- SDK packages: sdk/ts (@codegraff/sdk), sdk/py (codegraff) +- Lawplain uses graff: lawbook/ and root README positioning diff --git a/src/agent.zig b/src/agent.zig index fe8d4922..cb60e54a 100644 --- a/src/agent.zig +++ b/src/agent.zig @@ -35,6 +35,8 @@ const vision = @import("vision.zig"); const prompts = @import("prompts.zig"); const schema = @import("schema.zig"); const pricing = @import("pricing.zig"); +const models_cache = @import("models_cache.zig"); +const keys_cli = @import("keys_cli.zig"); const ansi = @import("ansi.zig"); const style = &ansi.style; @@ -72,6 +74,11 @@ fn compactTokenCount(buf: []u8, tokens: u64) []const u8 { std.fmt.bufPrint(buf, "{d}", .{tokens}) catch "?"; } +fn contextPercent(tokens: u64, window: u64) u64 { + if (window == 0) return 0; + return @min((tokens *| 100) / window, 100); +} + pub const TodoItem = struct { content: []const u8, status: []const u8, @@ -104,6 +111,10 @@ pub const Agent = struct { /// subagents/one-shots (they free their own arena per task); scratchAlloc() /// then falls back to arena, so their behavior is unchanged. scratch_arena: ?*std.heap.ArenaAllocator = null, + /// Optional allocator for a temporary request transaction. compact() points + /// this at its arena so failed summaries do not leak message rewrites or + /// partial response trees into the long-lived session arena. + message_mutation_arena: ?Allocator = null, io: Io, client: *std.http.Client, provider: Provider, @@ -141,6 +152,8 @@ pub const Agent = struct { snapshots: ?*tools_mod.Snapshots = null, // file-edit history for /rewind (root only) pending_image: ?vision.PendingImage = null, // staged by /image, sent with the next turn home: []const u8 = "", // $HOME, for /key persistence (set by main) + model_catalog: ?models_cache.LazyCodexCatalog = null, // demand-loaded dynamic Codex rows (root only) + stored_keys_loaded: bool = true, // false after an explicit-provider launch; model surfaces fill the remaining Keychain slots keep_context: bool = true, // carry the conversation across wire-format model switches (/keepcontext) reasoning: ReasoningEffort = .medium, // reasoning/thinking depth — codex, deepseek, codegraff (/effort, /reasoning) fast: bool = false, // codex "fast" mode → priority service_tier (/fast) @@ -167,13 +180,19 @@ pub const Agent = struct { strict: bool = false, completed: ?[]const u8 = null, last_context_tokens: u64 = 0, + context_local_tokens: u64 = 0, // local request estimate paired with last_context_tokens + last_usage_includes_output: bool = false, // fresh usage covers response items step* is about to append /// Detail of the most recent API error — carried into the --json `error` /// event, which otherwise only knows "api error". last_api_error: ?[]const u8 = null, /// Text streamed so far in the current request — on Esc-interrupt this is /// what survives into history (with an "[interrupted]" marker appended). partial_text: std.ArrayList(u8) = .empty, - stream_quiet: bool = false, // suppress live streaming (compaction summary) + stream_quiet: bool = false, // suppress live streaming (one-shot and internal requests) + compaction_request: bool = false, // the current model call is the synthetic compaction-summary request + last_request_context_overflow: bool = false, // explicit provider rejection, not an inferred meter threshold + last_request_write_failed: bool = false, // transport gave up specifically with WriteFailed this request + compact_transport_failures: u8 = 0, // bounded escape for repeated opaque over-cap WriteFailed/network failures ws_off: bool = false, // codex ws transport disabled for this session after a handshake/transport fallback to SSE (#codex-ws) streamed_text: bool = false, // the last request printed its text live thinking_open: bool = false, // a live "Thinking" reasoning block is currently streaming (/thinking) @@ -220,12 +239,13 @@ pub const Agent = struct { if (self.strict) try writePromptBadge(w, style.red, "Strict"); if (self.ultracode_mode) try writePromptBadge(w, style.magenta, "Ultracode"); try w.print("{s} · cwd {s}{s}{s}", .{ style.dim, style.reset, main_mod.g_cwd_display, style.dim }); + const context_tokens = self.effectiveContextTokens(); if (self.last_context_tokens > 0) { const threshold = self.provider.compactAt(); - const pct = if (self.provider.context > 0) self.last_context_tokens * 100 / self.provider.context else 0; + const pct = contextPercent(context_tokens, self.provider.context); var used_buf: [24]u8 = undefined; try w.print(" · {s}/{d}k ctx ({d}% · compact@{d}k){s}{s}]{s} {s}›{s} ", .{ - compactTokenCount(&used_buf, self.last_context_tokens), + compactTokenCount(&used_buf, context_tokens), self.provider.context / 1000, pct, threshold / 1000, @@ -242,6 +262,10 @@ pub const Agent = struct { } pub fn say(self: *Agent, comptime fmt: []const u8, args: anytype) !void { + // stdout is a strict JSONL transport in --json mode. Human-facing + // notices are represented by their structured terminal/error events; + // never leak an unframed line that breaks SDK parsers. + if (main_mod.json_mode and !self.sub) return; if (self.out) |w| { try w.print(fmt, args); try w.flush(); @@ -293,6 +317,47 @@ pub const Agent = struct { }; } + /// Root catalogs include meta + MCP tools and are much larger than the + /// subagent constants. Materialize only a wire format the session actually + /// uses; provider switching calls this before updating the context meter. + pub fn ensureRootTools(self: *Agent, kind: Provider.Kind) !void { + if (self.sub) return; + const slot = switch (kind) { + .anthropic => &self.tools_anthropic, + .openai => &self.tools_openai, + .responses => &self.tools_responses, + }; + if (slot.*.len != 0) return; + const specs = try schema.effectiveRootSpecs(self.arena); + const connected: []const mcp.Tool = if (self.registry) |registry| registry.tools else &.{}; + slot.* = try schema.renderRootTools(self.arena, kind, specs, connected); + } + + /// A live MCP registry change invalidates provider-specific catalogs. The + /// active format is immediately rebuilt; inactive formats remain lazy. + pub fn invalidateRootTools(self: *Agent) void { + if (self.sub) return; + self.tools_anthropic = ""; + self.tools_openai = ""; + self.tools_responses = ""; + } + + pub noinline fn ensureModelCatalog(self: *Agent, keys: provider_mod.Keys) void { + if (self.model_catalog) |*catalog| + catalog.ensure(self.io, self.gpa, self.arena, self.home, keys.get("codex") orelse "", keys.codex_account); + } + + pub fn reloadModelCatalog(self: *Agent, keys: provider_mod.Keys) void { + if (self.model_catalog) |*catalog| catalog.invalidate(); + self.ensureModelCatalog(keys); + } + + pub noinline fn ensureStoredKeys(self: *Agent, keys: *provider_mod.Keys) void { + if (self.stored_keys_loaded) return; + if (self.home.len != 0) keys_cli.loadMissingStoredKeys(self.io, self.gpa, self.arena, self.home, keys, .all); + self.stored_keys_loaded = true; + } + /// Run until the model stops (or, in strict mode, calls /// attempt_completion). Returns the final assistant text (arena-owned). /// Close the held codex Responses WS session and reset the delta state. Called @@ -311,6 +376,9 @@ pub const Agent = struct { } pub fn runTurn(self: *Agent) anyerror![]const u8 { + // Defensive for restored/embedded agents whose provider was assigned + // directly instead of going through providers.applyProvider. + try self.ensureRootTools(self.provider.kind); self.closeCodexWs(); // fresh codex WS session per turn defer self.closeCodexWs(); self.completed = null; @@ -342,7 +410,11 @@ pub const Agent = struct { // keyed to dropped messages (same closeCodexWs-after-trim reason as // the in-turn recovery at agent_request.zig:273). self.closeCodexWs(); - self.compactOrRecover(true); + // Match the between-turn policy: at the ordinary compactAt + // threshold, a transient/empty summary must not immediately drop + // real history. Destructive recovery is reserved for >=95%. + const recovery_meter = self.effectiveContextTokens(); + self.compactOrRecover(self.provider.nearContextLimit(recovery_meter)); self.closeCodexWs(); } const root = try self.request(self.toolsJson()); @@ -362,6 +434,12 @@ pub const Agent = struct { pub const request = @import("agent_request.zig").request; pub const inputOverCompactThreshold = @import("agent_request.zig").inputOverCompactThreshold; pub const fullInputEstimateTokens = @import("agent_request.zig").fullInputEstimateTokens; + pub const fullRequestEstimateTokens = @import("agent_request.zig").fullRequestEstimateTokens; + pub const contextEstimate = @import("agent_request.zig").contextEstimate; + pub const contextEstimateFromInputBytes = @import("agent_request.zig").contextEstimateFromInputBytes; + pub const effectiveContextTokens = @import("agent_request.zig").effectiveContextTokens; + pub const pairContextMeterWithCurrentLocal = @import("agent_request.zig").pairContextMeterWithCurrentLocal; + pub const rebaseContextMeter = @import("agent_request.zig").rebaseContextMeter; pub const recordUsage = @import("agent_request.zig").recordUsage; pub const usageInt = @import("agent_request.zig").usageInt; pub const recordCost = @import("agent_request.zig").recordCost; @@ -478,6 +556,10 @@ pub const Agent = struct { return if (self.scratch_arena) |sa| sa.allocator() else self.arena; } + pub fn messageMutationAlloc(self: *Agent) Allocator { + return self.message_mutation_arena orelse self.arena; + } + pub const escWatchTask = @import("agent_interrupt.zig").escWatchTask; pub const escPressed = @import("agent_interrupt.zig").escPressed; pub const drainSteerStdin = @import("agent_interrupt.zig").drainSteerStdin; @@ -549,3 +631,44 @@ test "compact token counts keep prompt usage readable" { try std.testing.expectEqualStrings("999", compactTokenCount(&buf, 999)); try std.testing.expectEqualStrings("138k", compactTokenCount(&buf, 138_082)); } + +test "context percent saturates malformed or over-window server meters" { + try std.testing.expectEqual(@as(u64, 0), contextPercent(100, 0)); + try std.testing.expectEqual(@as(u64, 50), contextPercent(50_000, 100_000)); + try std.testing.expectEqual(@as(u64, 100), contextPercent(150_000, 100_000)); + try std.testing.expectEqual(@as(u64, 100), contextPercent(std.math.maxInt(u64), 100_000)); +} + +test "lazy root tool catalogs preserve MCP tools across provider formats" { + var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator); + defer arena_state.deinit(); + const arena = arena_state.allocator(); + var registry = mcp.Registry.empty(std.testing.allocator, std.testing.io); + defer registry.deinit(); + var connected = [_]mcp.Tool{.{ + .server_index = 0, + .original_name = "search", + .qualified_name = "mcp__bench__search", + .description = "search the benchmark index", + .input_schema = .{ .object = .empty }, + }}; + registry.tools = &connected; + + var agent: Agent = undefined; + agent.arena = arena; + agent.sub = false; + agent.registry = ®istry; + agent.tools_anthropic = ""; + agent.tools_openai = ""; + agent.tools_responses = ""; + + try agent.ensureRootTools(.responses); + try std.testing.expect(std.mem.indexOf(u8, agent.tools_responses, "mcp__bench__search") != null); + try std.testing.expectEqual(@as(usize, 0), agent.tools_openai.len); + try std.testing.expectEqual(@as(usize, 0), agent.tools_anthropic.len); + + agent.invalidateRootTools(); + try agent.ensureRootTools(.anthropic); + try std.testing.expect(std.mem.indexOf(u8, agent.tools_anthropic, "mcp__bench__search") != null); + try std.testing.expectEqual(@as(usize, 0), agent.tools_responses.len); +} diff --git a/src/agent_compact.zig b/src/agent_compact.zig index 02b8dcb9..fcdaba2d 100644 --- a/src/agent_compact.zig +++ b/src/agent_compact.zig @@ -1,22 +1,14 @@ -//! Context management: client-side history compaction (compact/ -//! compactOrRecover) and emergency-trim recovery when compaction itself -//! can't run, plus the --eval/--until eval-driven loop (runEval/ -//! appendEvalLog) and its optional LLM-as-judge scorer (runJudge). Split -//! out of the Agent struct (#123, 600-line goal). +//! Context management: transactional history compaction and emergency +//! recovery when a summary request cannot run. const std = @import("std"); -const Io = std.Io; const Value = std.json.Value; const Allocator = std.mem.Allocator; const utf8Prefix = @import("util.zig").utf8Prefix; const main_mod = @import("main.zig"); const agent_mod = @import("agent.zig"); -const repl_glue = @import("repl_glue.zig"); const Agent = agent_mod.Agent; -const ToolCall = tools_mod.ToolCall; -const ExecResult = tools_mod.ExecResult; -const parseEvalScore = repl_glue.parseEvalScore; const prompts = @import("prompts.zig"); const compact_instruction = prompts.compact_instruction; @@ -26,249 +18,107 @@ const textMessage = messages_mod.textMessage; const title_mod = @import("title.zig"); const assistantText = title_mod.assistantText; -const scoring = @import("scoring.zig"); -const promptFingerprint = scoring.promptFingerprint; -const providerClass = scoring.providerClass; -const signScore = scoring.signScore; +const agent_eval = @import("agent_eval.zig"); +pub const runEval = agent_eval.runEval; +pub const appendEvalLog = agent_eval.appendEvalLog; +pub const runJudge = agent_eval.runJudge; -const telemetry = @import("telemetry.zig"); - -const jobs = @import("jobs.zig"); -const runCapped = jobs.runCapped; - -const tools_mod = @import("tools.zig"); -const ToolCtx = tools_mod.ToolCtx; -const ToolOutput = tools_mod.ToolOutput; - -const subagent = @import("subagent.zig"); -const judgeTask = subagent.judgeTask; - -const provider_mod = @import("provider.zig"); // codex-ws regression tests below -const Provider = provider_mod.Provider; -const ws = @import("ws.zig"); -const agent_ws = @import("agent_ws.zig"); // codexWsIdleExpired/codex_ws_idle_ms (#codex-ws) - -/// Run the configured --eval scoring command, append the result to the -/// scores log (.graff/eval-log.tsv), and return a verdict for the model: -/// score (0-100), best so far, target, and whether the target is met. The -/// harness runs the command, so the model cannot fake the number. (eval tool) -pub fn runEval(self: *Agent, note: []const u8) !ExecResult { - const cmd = self.eval_cmd orelse return .{ - .text = "no eval command configured - relaunch graff with --eval and --until , or ask the user to set one", - .is_error = true, - }; - const run = runCapped(self.gpa, self.io, &.{ "/bin/sh", "-c", cmd }, 64 * 1024, 16 * 1024, 0) catch |e| - return .{ .text = try std.fmt.allocPrint(self.arena, "eval command could not run: {t}", .{e}), .is_error = true }; - defer self.gpa.free(run.stdout); - defer self.gpa.free(run.stderr); - self.eval_iter += 1; - const exit_code: i32 = switch (run.term) { - .exited => |c| @intCast(c), - else => -1, - }; - const det = parseEvalScore(run.stdout) orelse parseEvalScore(run.stderr); - - // LLM-as-judge (--judge): an independent subagent inspects the actual - // artifacts against the rubric and returns its own 0-100 score. Both - // scores must clear the target, so the binding value is their min(). - // Skip the judge when the deterministic command itself produced no - // score - fix that first rather than burn a judge run. - const judge: ?f64 = if (self.eval_judge != null and det != null) self.runJudge(self.eval_judge.?, run.stdout, note) else null; - const combined: ?f64 = if (self.eval_judge == null) - det - else if (det != null and judge != null) - @min(det.?, judge.?) - else - null; - - const target_f: f64 = @floatFromInt(self.eval_target); - const improved = if (combined) |s| (self.eval_best < 0 or s > self.eval_best) else false; - if (combined) |s| { - if (self.eval_best < 0 or s > self.eval_best) self.eval_best = s; - } - const met = if (combined) |s| s >= target_f else false; - self.appendEvalLog(note, det, judge, combined, exit_code, met) catch {}; - - // Feed the eval-driven score into the fleet (docs/hyperagents.md §9.B): - // on a NEW BEST, submit the genome (this agent's persona) with its achieved - // score on the pinned eval set (the eval command's fingerprint). Without this - // the score only ever reached .graff/eval-log.tsv and the DGM/fleet never saw - // real eval-driven work — only darwincode/JSON-proto runs ever submitted. - if (combined) |s| { - if (improved and s > 0) { // s>0: skip the initial-state / total-failure 0 (don't pollute the cell mean) - if (s > 100) { - // Review F8: parseEvalScore does NOT bound its result (a stray - // "score: 9000" line parses as 9000) — an out-of-[0,100] score - // must never be signed or submitted. Skip the score AND its - // paired propose/submit, mirroring mainloop /score's explicit - // rejection, so the submit counter stays in sync with stored - // scores. The local eval verdict below still shows the number. - if (self.tracer) |tr| tr.note("fleet", "score skipped: eval score outside [0,100]"); - } else if (telemetry.g_telem) |t| { - const sys = self.systemPrompt(); - const genome_fp = promptFingerprint(sys); - const esh_fp = promptFingerprint(cmd); - const genome: []const u8 = &genome_fp; - const esh: []const u8 = &esh_fp; - const run_id: []const u8 = &scoring.g_run_id; - const pclass = providerClass(self.provider.model); - // --niche tags this score's cell. Without it the score lands in the - // anonymous "" niche, which pullElites can never match to a builtin — - // so an eval session that wants to grow a champion must name its role. - // Truncated to 64 chars and sanitized (tab/newline/CR → ' ', review - // F7) BEFORE signing (fleetEvent's own niche cap, same as mainloop - // /score) so signed bytes equal ingested bytes. - var niche_buf: [64]u8 = undefined; - const niche = scoring.sanitizeMetaField(&niche_buf, utf8Prefix(self.eval_niche, 64)); - // Genome-send (graff-dgm.md §B): the eval genome is this agent's own - // persona, never spawned via runSub, so its prompt_text never reached - // the worker. A cell only promotes when harness_scores joins to a - // harness_genomes row, so ride the genome text over on a `propose` - // (deduped by prompt_sha) before the score — else a winning eval cell - // has nothing to serve. Gated on a niche: a "" cell is unpromotable. - // Oversized genomes skip the propose (review F6): the server verifies - // the fingerprint over the carried text, so a truncated genome would - // be dropped there anyway. - if (niche.len > 0) { - if (sys.len <= telemetry.Telemetry.max_propose_text) - t.fleetEvent("propose", niche, genome, "", pclass, "", 0, sys) - else if (self.tracer) |tr| tr.note("fleet", "propose skipped: genome > 64KB"); - } - // SCORE SCALE CONTRACT (issue #168 Gap 4): local UX stays - // 0-100 (the /100 verdicts below, eval_best, eval_target), - // but every score that leaves the client is [0,1] — divide at - // the emission boundary; s01 is what gets signed (v2: niche + - // provider_class in the envelope) and sent. - const s01 = s / 100.0; - const sig = signScore(genome, "", s01, run_id, "", "", esh, niche, pclass); - const sig_s: []const u8 = if (scoring.g_score_key != null) &sig else ""; - var provbuf: [512]u8 = undefined; - const prov = std.fmt.bufPrint(&provbuf, "{s}\t{s}\t{s}\t{s}\t{s}", .{ "", "", esh, pclass, niche }) catch ""; - t.scoreEvent(genome, "", s01, run_id, sig_s, prov); - t.fleetEvent("submit", niche, genome, "", pclass, esh, 0, ""); - } - } - } - - var aw: Io.Writer.Allocating = .init(self.arena); - const w = &aw.writer; - if (combined) |s| { - if (self.eval_judge != null) { - try w.print("eval #{d}: deterministic {d:.1} + judge {d:.1} -> {d:.1}/100 (best {d:.1}, target {d}). ", .{ self.eval_iter, det.?, judge.?, s, self.eval_best, self.eval_target }); - } else { - try w.print("eval #{d}: score {d:.1}/100 (best {d:.1}, target {d}). ", .{ self.eval_iter, s, self.eval_best, self.eval_target }); - } - if (met) - try w.writeAll("TARGET MET - finish and report the final scores.") - else if (improved) - try w.writeAll("Improved - keep going: fix the next biggest failure with one focused change.") - else - try w.writeAll("No gain over the best - try a different change; do not build on a regression."); - } else if (self.eval_judge != null and det != null and judge == null) { - try w.print("eval #{d}: deterministic score {d:.1}/100, but the judge returned no parseable score (it may have errored). Re-run after checking the rubric. ", .{ self.eval_iter, det.? }); - } else { - try w.print("eval #{d}: command ran but no score parsed - print a bare number, or JSON with a score field (0-100 or 0-1), on the last line. ", .{self.eval_iter}); - } - const tail = if (run.stdout.len > 1500) run.stdout[run.stdout.len - 1500 ..] else run.stdout; - try w.print("\n[exit {d}] eval output (tail):\n{s}", .{ exit_code, tail }); - return .{ .text = try self.arena.dupe(u8, aw.writer.buffered()), .is_error = false }; -} - -/// Append one tab-separated row to the scores log (.graff/eval-log.tsv). -/// Best-effort - a failed write never breaks the loop. -pub fn appendEvalLog(self: *Agent, note: []const u8, det: ?f64, judge: ?f64, score: ?f64, exit_code: i32, met: bool) !void { - Io.Dir.cwd().createDir(self.io, ".graff", .default_dir) catch {}; - const path = ".graff/eval-log.tsv"; - const existing = Io.Dir.cwd().readFileAlloc(self.io, path, self.arena, .limited(2 * 1024 * 1024)) catch ""; - var aw: Io.Writer.Allocating = .init(self.arena); - const w = &aw.writer; - try w.writeAll(existing); - if (existing.len > 0 and existing[existing.len - 1] != '\n') try w.writeByte('\n'); - try w.print("iter={d}\tscore=", .{self.eval_iter}); - if (score) |s| try w.print("{d:.2}", .{s}) else try w.writeAll("NA"); - try w.writeAll("\tdet="); - if (det) |d| try w.print("{d:.2}", .{d}) else try w.writeAll("NA"); - try w.writeAll("\tjudge="); - if (judge) |j| try w.print("{d:.2}", .{j}) else try w.writeAll("NA"); - try w.print("\tbest={d:.2}\ttarget={d}\tmet={s}\texit={d}\tnote=", .{ self.eval_best, self.eval_target, if (met) "yes" else "no", exit_code }); - for (note) |ch| try w.writeByte(if (ch < 0x20) ' ' else ch); - try w.writeByte('\n'); - Io.Dir.cwd().writeFile(self.io, .{ .sub_path = path, .data = aw.writer.buffered() }) catch {}; -} - -/// Spawn an independent LLM judge (a read-only subagent) to score the -/// current work against the --judge rubric on a 0-100 scale. The judge -/// inspects the real artifacts with its own tools and ends its report with -/// a `score:` line, parsed the same way as a deterministic eval. Runs on a -/// pool thread via judgeTask (mirrors workflowTask) so the eval handler can -/// await it. Returns null if the judge could not run or gave no score. -pub fn runJudge(self: *Agent, rubric: []const u8, eval_output: []const u8, note: []const u8) ?f64 { - const ctx: ToolCtx = .{ - .gpa = self.gpa, - .io = self.io, - .client = self.client, - .provider = self.provider, - .registry = if (self.sub) null else self.registry, - .from_sub = self.sub, - .approvals = self.approvals, - .tracer = self.tracer, - .snapshots = self.snapshots, - .tools_used = &self.tools_used, +/// Ask the model for a context-handoff summary (no tools), then restart +/// history from that summary. +pub fn summaryResponseComplete(self: *Agent, root: std.json.ObjectMap) bool { + if (root.get("incomplete")) |value| if (value == .bool and value.bool) return false; + return switch (self.provider.kind) { + .responses => true, // parseResponses marks incomplete/missing terminals above + .anthropic => blk: { + // Raw non-streaming compatibility gateways may omit stop_reason. + // Stream reassembly marks that same absence as root.incomplete, so + // only an explicit truncating/unknown reason is rejected here. + const reason = root.get("stop_reason") orelse break :blk true; + if (reason == .null) break :blk true; + break :blk reason == .string and + (std.mem.eql(u8, reason.string, "end_turn") or std.mem.eql(u8, reason.string, "stop_sequence")); + }, + .openai => blk: { + const choices = root.get("choices") orelse break :blk false; + if (choices != .array or choices.array.items.len == 0) break :blk false; + const choice = choices.array.items[0]; + if (choice != .object) break :blk false; + // finish_reason is optional on otherwise-valid raw gateway JSON. + // Assembled streams without a terminal reason carry root.incomplete. + const reason = choice.object.get("finish_reason") orelse break :blk true; + if (reason == .null) break :blk true; + break :blk reason == .string and std.mem.eql(u8, reason.string, "stop"); + }, }; - const evidence = if (eval_output.len > 1200) eval_output[eval_output.len - 1200 ..] else eval_output; - const what = if (note.len > 0) note else "(no note given)"; - const judge_prompt = std.fmt.allocPrint(self.arena, - \\Score the current state of the work in this directory against the rubric below, on a 0-100 scale. - \\ - \\RUBRIC: - \\{s} - \\ - \\The author's note on the latest change: {s} - \\ - \\An automated check was also run; its output (tail) is below as evidence. Form your OWN independent judgement of how well the artifacts satisfy the rubric - do not simply echo the check: - \\--- - \\{s} - \\--- - \\ - \\Inspect the actual files and artifacts the work produced (read them with your tools; do not modify anything), then score how fully they satisfy the rubric. End your reply with a single final line `score: ` where N is an integer from 0 to 100. - , .{ rubric, what, evidence }) catch return null; - var fut: Io.Future(ToolOutput) = self.io.async(judgeTask, .{ ctx, judge_prompt }); - const out = fut.await(self.io); - defer self.gpa.free(out.text); - if (out.is_error) return null; - return parseEvalScore(out.text); } -/// Ask the model for a context-handoff summary (no tools), then restart -/// history from that summary. pub fn compact(self: *Agent) anyerror!usize { if (self.messages.items.len == 0) { if (!main_mod.json_mode) try self.say("nothing to compact\n", .{}); return 0; } - if (!main_mod.json_mode) try self.say("[compacting ~{d} tokens…]\n", .{self.last_context_tokens}); + self.last_request_context_overflow = false; + if (!main_mod.json_mode) try self.say("[compacting ~{d} tokens…]\n", .{self.effectiveContextTokens()}); // #163: reclaim room BEFORE the summarization request so it fits under the // model's input cap. On codex/gpt-5.x an over-cap request fails to WRITE // (WriteFailed) rather than returning a clean overflow, so compaction could // never run once near the cap. Old tool outputs are superseded by the summary // anyway; truncating them keeps the request sendable + all pairing intact. - // #174: on the Responses path, prior-turn reasoning items go first — they - // dominate the resend bloat on long high-effort sessions, and the backend - // itself discards them from chained context, so dropping them can't lose - // anything the server would have kept. + // Build the summary request on a temporary deep copy of every JSON + // container. Pruning reasoning and truncating tool output must be + // transactional: if the summary call fails or returns empty, the live + // conversation remains byte-for-byte usable for the next attempt. + var compact_arena_state = std.heap.ArenaAllocator.init(self.gpa); + defer compact_arena_state.deinit(); + const compact_arena = compact_arena_state.allocator(); + const live_messages = self.messages; + const live_context_tokens = self.last_context_tokens; + const live_context_local_tokens = self.context_local_tokens; + const live_effective_context = self.effectiveContextTokens(); + self.messages = try cloneJsonArray(compact_arena, live_messages); + var installed_summary = false; + defer if (!installed_summary) { + self.messages = live_messages; + if (self.last_request_context_overflow) { + self.context_local_tokens = self.fullRequestEstimateTokens(); + self.last_context_tokens = @max(live_effective_context, self.provider.context); + } else { + // Summary construction and usage sampling are transactional too. + self.last_context_tokens = live_context_tokens; + self.context_local_tokens = live_context_local_tokens; + } + }; + + try self.messages.append(try textMessage(compact_arena, "user", compact_instruction)); + // #174: establish the synthetic summary turn before pruning Responses + // reasoning. An active tool loop's reasoning is newer than the real user + // turn and must remain while that loop is in flight, but it becomes prior- + // turn context once this synthetic user request is appended. Pruning first + // left precisely that large encrypted blob in the full summary resend. _ = dropPriorTurnReasoning(self); - _ = trimOldestToolOutputs(self); - try self.messages.append(try textMessage(self.arena, "user", compact_instruction)); - errdefer _ = self.messages.pop(); + _ = trimOldestToolOutputsAlloc(self, compact_arena); // The handoff summary is internal — don't stream it to the terminal. + const was_quiet = self.stream_quiet; + const was_compaction_request = self.compaction_request; + const was_message_mutation_arena = self.message_mutation_arena; self.stream_quiet = true; - defer self.stream_quiet = false; + self.compaction_request = true; + self.message_mutation_arena = compact_arena; + defer self.stream_quiet = was_quiet; + defer self.compaction_request = was_compaction_request; + defer self.message_mutation_arena = was_message_mutation_arena; const root = try self.request(null); - const summary = assistantText(self.provider.kind, root); + // Any complete transport response proves the previous opaque failure was + // transient/non-wedging, even when its summary text is empty or truncated. + self.compact_transport_failures = 0; + if (!summaryResponseComplete(self, root)) { + if (!main_mod.json_mode) try self.say("[compaction failed: provider returned an incomplete summary, history unchanged]\n", .{}); + return error.IncompleteSummary; + } + const summary = std.mem.trim(u8, assistantText(self.provider.kind, root), " \t\r\n"); if (summary.len == 0) { if (!main_mod.json_mode) try self.say("[compaction failed: empty summary, history unchanged]\n", .{}); - _ = self.messages.pop(); return error.EmptySummary; } @@ -284,10 +134,37 @@ pub fn compact(self: *Agent) anyerror!usize { try fresh.append(try textMessage(self.arena, "user", handoff)); self.messages = fresh; self.last_context_tokens = 0; + self.context_local_tokens = 0; + installed_summary = true; if (!main_mod.json_mode) try self.say("[history compacted to a {d}-char summary]\n", .{summary.len}); return summary.len; } +/// Clone JSON arrays/objects while borrowing immutable leaf strings. Send-time +/// normalization replaces Value slots rather than mutating string bytes, so +/// separate containers are sufficient to isolate a compaction request without +/// duplicating multi-megabyte reasoning and tool-output payloads. +fn cloneJsonValue(arena: Allocator, value: Value) Allocator.Error!Value { + return switch (value) { + .array => |src| .{ .array = try cloneJsonArray(arena, src) }, + .object => |src| blk: { + var out: std.json.ObjectMap = .empty; + var it = src.iterator(); + while (it.next()) |entry| + try out.put(arena, entry.key_ptr.*, try cloneJsonValue(arena, entry.value_ptr.*)); + break :blk .{ .object = out }; + }, + else => value, + }; +} + +pub fn cloneJsonArray(arena: Allocator, src: std.json.Array) Allocator.Error!std.json.Array { + var out = std.json.Array.init(arena); + try out.ensureTotalCapacity(src.items.len); + for (src.items) |item| try out.append(try cloneJsonValue(arena, item)); + return out; +} + pub fn cleanUserTurn(m: Value) bool { if (m != .object) return false; const role = m.object.get("role") orelse return false; @@ -340,36 +217,6 @@ pub fn dropPriorTurnReasoning(self: *Agent) usize { return dropped; } -test "dropPriorTurnReasoning (#174): prior-turn reasoning goes, current turn + non-responses stay" { - var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator); - defer arena_state.deinit(); - const a = arena_state.allocator(); - - var msgs = std.json.Array.init(a); - try msgs.append(try textMessage(a, "user", "turn one")); - try msgs.append(try std.json.parseFromSliceLeaky(Value, a, "{\"type\":\"reasoning\",\"encrypted_content\":\"OLD1\"}", .{})); - try msgs.append(try textMessage(a, "assistant", "reply one")); - try msgs.append(try std.json.parseFromSliceLeaky(Value, a, "{\"type\":\"reasoning\",\"encrypted_content\":\"OLD2\"}", .{})); - try msgs.append(try textMessage(a, "user", "turn two")); - try msgs.append(try std.json.parseFromSliceLeaky(Value, a, "{\"type\":\"reasoning\",\"encrypted_content\":\"CURRENT\"}", .{})); - try msgs.append(try std.json.parseFromSliceLeaky(Value, a, "{\"type\":\"function_call\",\"name\":\"bash\",\"call_id\":\"c1\",\"arguments\":\"{}\"}", .{})); - - var agent: Agent = undefined; - agent.provider = .{ .id = "codex", .kind = .responses, .auth = .bearer, .url = "", .api_key = "", .model = "gpt-5", .context = 100_000 }; - agent.messages = msgs; - - try std.testing.expectEqual(@as(usize, 2), dropPriorTurnReasoning(&agent)); - try std.testing.expectEqual(@as(usize, 5), agent.messages.items.len); - // current-turn reasoning (after the last user message) survives, in order - const kept = agent.messages.items[3].object.get("encrypted_content").?.string; - try std.testing.expectEqualStrings("CURRENT", kept); - - // non-responses providers are untouched - agent.provider.kind = .openai; - try std.testing.expectEqual(@as(usize, 0), dropPriorTurnReasoning(&agent)); - try std.testing.expectEqual(@as(usize, 5), agent.messages.items.len); -} - /// Index to cut history at for an emergency trim: the first clean user turn /// at or after the midpoint, so messages[cut..] is always a valid /// conversation start (never an orphaned tool_result). null when there is no @@ -433,13 +280,24 @@ fn truncateToolOutput(arena: Allocator, m: *Value, cap: usize, note: []const u8) return 0; } +/// Re-pair the meter after removing locally measurable context. The server-only +/// delta remains intact, while the current local component reflects the trim. +fn accountForReclaimedTokens(self: *Agent, reclaimed_tokens: u64) void { + if (reclaimed_tokens == 0) return; + self.rebaseContextMeter(); +} + +fn accountForReclaimedContext(self: *Agent, reclaimed: usize) void { + accountForReclaimedTokens(self, @intCast(reclaimed / 4)); +} + /// #163: reclaim context when it has overflowed and there is no clean user turn /// to cut at (a runaway tool loop is all tool_call/tool_result after the last /// user turn). Truncate the OLDEST tool outputs in place, keeping the most recent /// `keep_recent` verbatim (opencode-style) and every call/output pair intact /// (codex's trim_function_call_history). Never drops a message, so no orphaned /// tool_result can reach the API. Returns bytes reclaimed. -pub fn trimOldestToolOutputs(self: *Agent) usize { +pub fn trimOldestToolOutputsAlloc(self: *Agent, arena: Allocator) usize { const keep_recent: usize = 4; const stub_cap: usize = 400; var total: usize = 0; @@ -453,13 +311,16 @@ pub fn trimOldestToolOutputs(self: *Agent) usize { if (!isToolOutputMsg(m.*)) continue; seen += 1; if (seen > total - keep_recent) break; // keep the most recent verbatim - reclaimed += truncateToolOutput(self.arena, m, stub_cap, "[old tool output truncated to recover context (#163)]"); + reclaimed += truncateToolOutput(arena, m, stub_cap, "[old tool output truncated to recover context (#163)]"); } - // #202: reflect the trimmed size instead of blinding the meter to 0. - if (reclaimed > 0) self.last_context_tokens = self.fullInputEstimateTokens(); + accountForReclaimedContext(self, reclaimed); return reclaimed; } +pub fn trimOldestToolOutputs(self: *Agent) usize { + return trimOldestToolOutputsAlloc(self, self.arena); +} + /// #193 follow-up: bound ANY single tool output to `cap` serialized bytes before /// send, in place, across every wire format (responses `function_call_output`, /// openai `role:"tool"`, anthropic `tool_result` blocks). normalizeResponsesHistory @@ -475,25 +336,31 @@ pub fn capOversizedToolOutputs(self: *Agent, cap: usize) usize { var reclaimed: usize = 0; for (self.messages.items) |*m| { if (isToolOutputMsg(m.*)) - reclaimed += truncateToolOutput(self.arena, m, cap, "[tool output truncated: over this model's per-result cap — read/fetch a smaller range (#193)]"); + reclaimed += truncateToolOutput(self.messageMutationAlloc(), m, cap, "[tool output truncated: over this model's per-result cap — read/fetch a smaller range (#193)]"); } - // #202: reflect the trimmed size instead of blinding the meter to 0, so the - // between-turns gate keeps working and an overflow recover-pin isn't clobbered. - if (reclaimed > 0) self.last_context_tokens = self.fullInputEstimateTokens(); + // These outputs are appended after the prior response's usage was recorded, + // so reclaimed bytes were never part of that authoritative server reading. + // Never subtract them from it; only raise the floor to the now-capped full + // request estimate. + if (reclaimed > 0) self.rebaseContextMeter(); return reclaimed; } /// Last-resort context recovery when compact() itself can't run — typically /// because the history already overflows the window, so the summarization /// request overflows too and fails. Drops the oldest messages at a safe -/// boundary; returns the count dropped (0 if none). The next turn re-measures -/// context from the provider usage. +/// boundary; returns the count dropped (0 if none). Conservatively reduce the +/// authoritative meter only by the locally measurable reclaimed tokens: hidden +/// server-side reasoning may make the true reduction larger, but must never let +/// a partial trim blind the next pre-send gate. pub fn emergencyTrim(self: *Agent) usize { if (emergencyCutIndex(self.messages.items)) |cut| { + const before_tokens = self.fullInputEstimateTokens(); var fresh = std.json.Array.init(self.arena); for (self.messages.items[cut..]) |m| fresh.append(m) catch return 0; self.messages = fresh; - self.last_context_tokens = 0; + const after_tokens = self.fullInputEstimateTokens(); + accountForReclaimedTokens(self, before_tokens -| after_tokens); return cut; } // #163: no clean user turn to cut at (a runaway tool loop). Don't wedge the @@ -508,11 +375,32 @@ pub fn emergencyTrim(self: *Agent) usize { /// later turn failed at the same huge token count (issue #88). Surface the /// failure and, when `trim_on_fail`, emergency-trim so the next turn has /// room. Best-effort; never throws into the REPL loop. +pub fn repeatedOpaqueCompactionFailure(self: *Agent, err: anyerror) bool { + const opaque_transport = err == error.ApiError and self.last_request_write_failed; + if (opaque_transport) + self.compact_transport_failures +|= 1 + else + self.compact_transport_failures = 0; + const threshold = self.provider.compactAt(); + const effective = self.effectiveContextTokens(); + const locally_over_window = self.provider.context > 0 and self.fullRequestEstimateTokens() >= self.provider.context; + return opaque_transport and + threshold > 0 and + self.compact_transport_failures >= 2 and + (self.provider.nearContextLimit(effective) or locally_over_window); +} + pub fn compactOrRecover(self: *Agent, trim_on_fail: bool) void { - if (self.compact()) |_| return else |err| { + if (self.compact()) |_| { + self.compact_transport_failures = 0; + return; + } else |err| { switch (err) { - error.Interrupted => return, // user hit Esc mid-compaction - error.EmptySummary => {}, // compact() already explained it + error.Interrupted => { + self.compact_transport_failures = 0; + return; // user hit Esc mid-compaction + }, + error.EmptySummary, error.IncompleteSummary => {}, // compact() already explained it else => { if (main_mod.json_mode) self.emit(.{ .type = "error", .message = std.fmt.allocPrint(self.arena, "auto-compaction failed: {s}", .{@errorName(err)}) catch "auto-compaction failed" }) @@ -520,9 +408,16 @@ pub fn compactOrRecover(self: *Agent, trim_on_fail: bool) void { self.say("[auto-compaction failed: {t}]\n", .{err}) catch {}; }, } - if (!trim_on_fail) return; + const repeated_opaque_overflow = repeatedOpaqueCompactionFailure(self, err); + // The caller's policy is computed before compact() makes its summary + // request. Override it only for a concrete provider overflow rejection, + // or after two consecutive WriteFailed compaction attempts when the + // effective meter is near 95% (or local bytes prove over-window). The + // first failure and ordinary transport outages always preserve history. + if (!trim_on_fail and !self.last_request_context_overflow and !repeated_opaque_overflow) return; const dropped = self.emergencyTrim(); if (dropped > 0) { + self.compact_transport_failures = 0; if (main_mod.json_mode) self.emit(.{ .type = "compact", .ok = true, .trimmed = dropped }) else @@ -532,243 +427,3 @@ pub fn compactOrRecover(self: *Agent, trim_on_fail: bool) void { } } } - -test "trimOldestToolOutputs recovers a runaway tool-loop history (#163)" { - var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator); - defer arena_state.deinit(); - const a = arena_state.allocator(); - var msgs = std.json.Array.init(a); - try msgs.append(try textMessage(a, "user", "babysit the CI")); // the only clean user turn - const big = try a.alloc(u8, 5000); - @memset(big, 'x'); - var i: usize = 0; - while (i < 10) : (i += 1) { - var o: std.json.ObjectMap = .empty; - try o.put(a, "type", .{ .string = "function_call_output" }); - try o.put(a, "call_id", .{ .string = "c" }); - try o.put(a, "output", .{ .string = big }); - try msgs.append(.{ .object = o }); - } - var agent: Agent = undefined; - agent.messages = msgs; - agent.arena = a; - agent.last_context_tokens = 100000; - // no clean user turn after the midpoint -> the old emergencyTrim would give up - try std.testing.expect(emergencyCutIndex(agent.messages.items) == null); - const reclaimed = trimOldestToolOutputs(&agent); - try std.testing.expect(reclaimed > 0); // recovered instead of wedging - // #202: re-measured to the trimmed size instead of blinding the meter to 0 - try std.testing.expect(agent.last_context_tokens > 0); - try std.testing.expectEqual(agent.fullInputEstimateTokens(), agent.last_context_tokens); - var truncated: usize = 0; - var full: usize = 0; - for (agent.messages.items) |m| { - if (m == .object) if (m.object.get("output")) |out| if (out == .string) { - if (out.string.len < 1000) truncated += 1 else full += 1; - }; - } - try std.testing.expectEqual(@as(usize, 6), truncated); // 10 outputs, oldest 6 truncated - try std.testing.expectEqual(@as(usize, 4), full); // 4 most-recent kept verbatim - try std.testing.expectEqual(@as(usize, 11), agent.messages.items.len); // no message dropped -} - -test "capOversizedToolOutputs (#193): bounds an oversized output in every wire format, leaves small ones + non-tool msgs" { - var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator); - defer arena_state.deinit(); - const a = arena_state.allocator(); - const cap: usize = 1024; - const big = try a.alloc(u8, 8192); - @memset(big, 'x'); - - var msgs = std.json.Array.init(a); - // responses function_call_output (oversized) - var fco: std.json.ObjectMap = .empty; - try fco.put(a, "type", .{ .string = "function_call_output" }); - try fco.put(a, "call_id", .{ .string = "c1" }); - try fco.put(a, "output", .{ .string = big }); - try msgs.append(.{ .object = fco }); - // openai role:"tool" (oversized) - var tool: std.json.ObjectMap = .empty; - try tool.put(a, "role", .{ .string = "tool" }); - try tool.put(a, "tool_call_id", .{ .string = "c2" }); - try tool.put(a, "content", .{ .string = big }); - try msgs.append(.{ .object = tool }); - // anthropic user turn carrying a tool_result block (oversized) - var user: std.json.ObjectMap = .empty; - try user.put(a, "role", .{ .string = "user" }); - var blocks = std.json.Array.init(a); - var tr: std.json.ObjectMap = .empty; - try tr.put(a, "type", .{ .string = "tool_result" }); - try tr.put(a, "tool_use_id", .{ .string = "c3" }); - try tr.put(a, "content", .{ .string = big }); - try blocks.append(.{ .object = tr }); - try user.put(a, "content", .{ .array = blocks }); - try msgs.append(.{ .object = user }); - // a small tool output (within cap) — must be left untouched - var small: std.json.ObjectMap = .empty; - try small.put(a, "type", .{ .string = "function_call_output" }); - try small.put(a, "output", .{ .string = "ok" }); - try msgs.append(.{ .object = small }); - // a plain assistant text message — never a tool output, left untouched - try msgs.append(try textMessage(a, "assistant", "hello")); - - var agent: Agent = undefined; - agent.arena = a; - agent.messages = msgs; - agent.last_context_tokens = 42; - - const reclaimed = capOversizedToolOutputs(&agent, cap); - try std.testing.expect(reclaimed > 0); - // #202: re-measured to the trimmed size instead of blinding the meter to 0 - try std.testing.expect(agent.last_context_tokens > 0); - try std.testing.expectEqual(agent.fullInputEstimateTokens(), agent.last_context_tokens); - - // every oversized tool output is now within the cap, with a marker - const out0 = agent.messages.items[0].object.get("output").?.string; - try std.testing.expect(out0.len <= cap); - try std.testing.expect(std.mem.indexOf(u8, out0, "truncated") != null); - const out1 = agent.messages.items[1].object.get("content").?.string; - try std.testing.expect(out1.len <= cap); - const block = agent.messages.items[2].object.get("content").?.array.items[0]; - try std.testing.expect(block.object.get("content").?.string.len <= cap); - // within-cap output and the non-tool message are untouched - try std.testing.expectEqualStrings("ok", agent.messages.items[3].object.get("output").?.string); - try std.testing.expectEqualStrings("hello", agent.messages.items[4].object.get("content").?.string); - // cap == 0 disables the cap entirely (unknown window) - try std.testing.expectEqual(@as(usize, 0), capOversizedToolOutputs(&agent, 0)); -} - -test "cleanUserTurn: plain user text yes; assistant/tool_result no" { - var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator); - defer arena_state.deinit(); - const a = arena_state.allocator(); - try std.testing.expect(cleanUserTurn(try textMessage(a, "user", "hello"))); - try std.testing.expect(!cleanUserTurn(try textMessage(a, "assistant", "hi"))); - // an anthropic tool_result-only user message is NOT a clean conversation start - const tr = try std.json.parseFromSliceLeaky(Value, a, "{\"role\":\"user\",\"content\":[{\"type\":\"tool_result\",\"content\":\"ok\"}]}", .{}); - try std.testing.expect(!cleanUserTurn(tr)); -} - -test "closeCodexWs resets the delta session state + frees the response id (codex-ws)" { - var agent: Agent = undefined; - agent.gpa = std.testing.allocator; - agent.codex_ws = null; // no live WsClient to deinit in a unit test - agent.codex_prev_id = try std.testing.allocator.dupe(u8, "resp_abc123"); // must be freed (leak-checked) - agent.codex_sent_upto = 5; - agent.closeCodexWs(); - try std.testing.expect(agent.codex_prev_id == null); // freed + nulled - try std.testing.expectEqual(@as(usize, 0), agent.codex_sent_upto); // delta boundary reset - try std.testing.expect(agent.codex_ws == null); -} - -// (#codex-ws) The delta-body detection string check in agent_ws.zig's -// postLive() gates the WS-reanchor path (never SSE-replay a delta): it -// looks for the literal `"previous_response_id"` key that buildBody's -// .responses branch emits. Pin the exact substring so the two stay in -// sync — a future rename of the JSON key on either side breaks this test -// instead of silently reopening the SSE-replay bug. -test "postLive's delta-body detection matches the key buildBody emits (codex-ws)" { - const delta_body = "{\"model\":\"gpt-5\",\"previous_response_id\":\"resp_1\",\"input\":[]}"; - const full_body = "{\"model\":\"gpt-5\",\"input\":[]}"; - try std.testing.expect(std.mem.indexOf(u8, delta_body, "\"previous_response_id\"") != null); - try std.testing.expect(std.mem.indexOf(u8, full_body, "\"previous_response_id\"") == null); -} - -// (#codex-ws) The preemptive idle re-anchor decision (postResponsesWs closes a -// held WS the server has likely already killed instead of eating a failed -// round trip). Pure helper so no socket is needed: exactly-at-limit must NOT -// expire (only strictly past it), and a WS used moments ago must survive. -// opencode pools at 5 min; ours defaults to 4 (the backend killed a real -// session within 8.5 min idle). -test "codexWsIdleExpired: fires only strictly past the idle limit (codex-ws)" { - const limit = agent_ws.codex_ws_idle_ms; - try std.testing.expectEqual(@as(i64, 4 * std.time.ms_per_min), limit); // default: stay under the observed server kill - try std.testing.expect(!agent_ws.codexWsIdleExpired(1_000_000, 1_000_000)); // just used - try std.testing.expect(!agent_ws.codexWsIdleExpired(1_000_000 + limit, 1_000_000)); // exactly at the limit — keep - try std.testing.expect(agent_ws.codexWsIdleExpired(1_000_000 + limit + 1, 1_000_000)); // past it — re-anchor - try std.testing.expect(agent_ws.codexWsIdleExpired(1_000_000 + 510 * std.time.ms_per_s, 1_000_000)); // the real 8.5-min trace gap -} - -// (#codex-ws) End-to-end regression for the reanchor fix: buildBody's -// .responses branch must emit previous_response_id + a message-slice delta -// while a WS session + prev id are held, and after closeCodexWs (called by -// postLive on a delta transport error, and again by request()'s -// error.CodexWsReanchor handler) a rebuilt body must carry the FULL -// message history with no previous_response_id — never a stale delta -// replayed against a dead session. -test "buildBody (.responses): delta while WS live; full input after closeCodexWs (codex-ws)" { - var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator); - defer arena_state.deinit(); - const a = arena_state.allocator(); - - var msgs = std.json.Array.init(a); - try msgs.append(try textMessage(a, "user", "first")); - try msgs.append(try textMessage(a, "assistant", "second")); - try msgs.append(try textMessage(a, "user", "third — not yet sent")); - - var dummy_ws: ws.WsClient = undefined; // buildBody only checks != null, never dereferences - var agent: Agent = .{ - .gpa = std.testing.allocator, - .arena = a, - .io = undefined, // unused by buildBody - .client = undefined, // unused by buildBody - .provider = .{ .id = "codex", .kind = .responses, .auth = .bearer, .url = "https://x/responses", .api_key = "k", .model = "gpt-5", .context = 100_000 }, - .messages = msgs, - .sub = false, - .label = "", - .out = null, - .codex_ws = &dummy_ws, - .codex_prev_id = try std.testing.allocator.dupe(u8, "resp_live"), - .codex_sent_upto = 2, // server already holds messages[0..2]; delta = [2..] - }; - - const delta_body = try agent.buildBody(null, false, false, false); - defer std.testing.allocator.free(delta_body); - try std.testing.expect(std.mem.indexOf(u8, delta_body, "\"previous_response_id\":\"resp_live\"") != null); - try std.testing.expect(std.mem.indexOf(u8, delta_body, "third — not yet sent") != null); - try std.testing.expect(std.mem.indexOf(u8, delta_body, "\"first\"") == null); // NOT resent — already on the server - - // Simulate closeCodexWs's effect (covered by its own unit test above) - // without invoking it directly: it would call ws.WsClient.deinit on - // codex_ws, which sends a real close frame over `dummy_ws`'s - // uninitialized io/fd — fine for a live connection, unsafe for this - // struct-literal stand-in. What matters here is buildBody's behavior - // given the post-close state postLive/request() leave it in. - std.testing.allocator.free(agent.codex_prev_id.?); - agent.codex_ws = null; - agent.codex_prev_id = null; - agent.codex_sent_upto = 0; - const rebuilt_body = try agent.buildBody(null, false, false, false); - defer std.testing.allocator.free(rebuilt_body); - try std.testing.expect(std.mem.indexOf(u8, rebuilt_body, "\"previous_response_id\"") == null); - try std.testing.expect(std.mem.indexOf(u8, rebuilt_body, "\"first\"") != null); // full history restored - try std.testing.expect(std.mem.indexOf(u8, rebuilt_body, "third — not yet sent") != null); -} - -test "emergencyCutIndex: cuts at a clean user turn at/after the midpoint" { - var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator); - defer arena_state.deinit(); - const a = arena_state.allocator(); - var items = std.json.Array.init(a); - const roles = [_][]const u8{ "user", "assistant", "user", "assistant", "user", "assistant", "user", "assistant" }; - for (roles) |r| try items.append(try textMessage(a, r, "x")); - try std.testing.expectEqual(@as(?usize, 4), emergencyCutIndex(items.items)); // midpoint 4 is a user turn - try std.testing.expectEqual(@as(?usize, null), emergencyCutIndex(items.items[0..3])); // too short to trim -} - -test "emergencyCutIndex: skips a tool_result user message at the midpoint" { - var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator); - defer arena_state.deinit(); - const a = arena_state.allocator(); - var items = std.json.Array.init(a); - try items.append(try textMessage(a, "user", "x")); // 0 - try items.append(try textMessage(a, "assistant", "x")); // 1 - try items.append(try textMessage(a, "user", "x")); // 2 - try items.append(try textMessage(a, "assistant", "x")); // 3 - // 4: an anthropic tool_result-only user message (not a valid conversation start) - try items.append(try std.json.parseFromSliceLeaky(Value, a, "{\"role\":\"user\",\"content\":[{\"type\":\"tool_result\",\"content\":\"x\"}]}", .{})); // 4 (skip) - try items.append(try textMessage(a, "assistant", "x")); // 5 - try items.append(try textMessage(a, "user", "x")); // 6 (first clean user >= midpoint) - try items.append(try textMessage(a, "assistant", "x")); // 7 - try std.testing.expectEqual(@as(?usize, 6), emergencyCutIndex(items.items)); -} diff --git a/src/agent_compact_test.zig b/src/agent_compact_test.zig new file mode 100644 index 00000000..1c9bf340 --- /dev/null +++ b/src/agent_compact_test.zig @@ -0,0 +1,377 @@ +//! Focused regression tests for transactional compaction and context recovery. + +const std = @import("std"); +const Value = std.json.Value; +const Agent = @import("agent.zig").Agent; +const textMessage = @import("messages.zig").textMessage; +const compact_instruction = @import("prompts.zig").compact_instruction; + +const compact = @import("agent_compact.zig"); +const summaryResponseComplete = compact.summaryResponseComplete; +const cloneJsonArray = compact.cloneJsonArray; +const dropPriorTurnReasoning = compact.dropPriorTurnReasoning; +const trimOldestToolOutputsAlloc = compact.trimOldestToolOutputsAlloc; +const repeatedOpaqueCompactionFailure = compact.repeatedOpaqueCompactionFailure; +const trimOldestToolOutputs = compact.trimOldestToolOutputs; +const capOversizedToolOutputs = compact.capOversizedToolOutputs; +const cleanUserTurn = compact.cleanUserTurn; +const emergencyCutIndex = compact.emergencyCutIndex; +const emergencyTrim = compact.emergencyTrim; + +test "compaction accepts only complete provider terminal states" { + var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator); + defer arena_state.deinit(); + const a = arena_state.allocator(); + var agent: Agent = undefined; + + agent.provider = .{ .id = "codex", .kind = .responses, .auth = .bearer, .url = "", .api_key = "", .model = "gpt-5", .context = 100_000 }; + const responses_ok = try std.json.parseFromSliceLeaky(Value, a, "{}", .{}); + const responses_partial = try std.json.parseFromSliceLeaky(Value, a, "{\"incomplete\":true}", .{}); + try std.testing.expect(summaryResponseComplete(&agent, responses_ok.object)); + try std.testing.expect(!summaryResponseComplete(&agent, responses_partial.object)); + + agent.provider.kind = .anthropic; + const anthropic_ok = try std.json.parseFromSliceLeaky(Value, a, "{\"stop_reason\":\"end_turn\"}", .{}); + const anthropic_short = try std.json.parseFromSliceLeaky(Value, a, "{\"stop_reason\":\"max_tokens\"}", .{}); + const anthropic_compat = try std.json.parseFromSliceLeaky(Value, a, "{}", .{}); + try std.testing.expect(summaryResponseComplete(&agent, anthropic_ok.object)); + try std.testing.expect(!summaryResponseComplete(&agent, anthropic_short.object)); + try std.testing.expect(summaryResponseComplete(&agent, anthropic_compat.object)); + + agent.provider.kind = .openai; + const openai_ok = try std.json.parseFromSliceLeaky(Value, a, "{\"choices\":[{\"finish_reason\":\"stop\"}]}", .{}); + const openai_short = try std.json.parseFromSliceLeaky(Value, a, "{\"choices\":[{\"finish_reason\":\"length\"}]}", .{}); + const openai_compat = try std.json.parseFromSliceLeaky(Value, a, "{\"choices\":[{}]}", .{}); + const openai_stream_partial = try std.json.parseFromSliceLeaky(Value, a, "{\"incomplete\":true,\"choices\":[{\"finish_reason\":\"stop\"}]}", .{}); + try std.testing.expect(summaryResponseComplete(&agent, openai_ok.object)); + try std.testing.expect(!summaryResponseComplete(&agent, openai_short.object)); + try std.testing.expect(summaryResponseComplete(&agent, openai_compat.object)); + try std.testing.expect(!summaryResponseComplete(&agent, openai_stream_partial.object)); +} + +test "compaction working copy isolates reasoning and tool-output pruning" { + var live_arena_state = std.heap.ArenaAllocator.init(std.testing.allocator); + defer live_arena_state.deinit(); + const live_arena = live_arena_state.allocator(); + + var live = std.json.Array.init(live_arena); + try live.append(try textMessage(live_arena, "user", "run the tool loop")); + try live.append(try std.json.parseFromSliceLeaky(Value, live_arena, "{\"type\":\"reasoning\",\"encrypted_content\":\"KEEP\"}", .{})); + const big = try live_arena.alloc(u8, 5000); + @memset(big, 'x'); + for (0..10) |i| { + var output: std.json.ObjectMap = .empty; + try output.put(live_arena, "type", .{ .string = "function_call_output" }); + try output.put(live_arena, "call_id", .{ .string = try std.fmt.allocPrint(live_arena, "c{d}", .{i}) }); + try output.put(live_arena, "output", .{ .string = big }); + try live.append(.{ .object = output }); + } + + var work_arena_state = std.heap.ArenaAllocator.init(std.testing.allocator); + defer work_arena_state.deinit(); + const work_arena = work_arena_state.allocator(); + var agent: Agent = undefined; + agent.arena = work_arena; + agent.messages = try cloneJsonArray(work_arena, live); + agent.provider = .{ .id = "codex", .kind = .responses, .auth = .bearer, .url = "", .api_key = "", .model = "gpt-5", .context = 100_000 }; + agent.last_context_tokens = 90_000; + agent.sub = false; + agent.strict = false; + agent.sys_normal = ""; + agent.sys_strict = ""; + agent.tools_responses = ""; + agent.context_local_tokens = agent.fullRequestEstimateTokens(); + + try agent.messages.append(try textMessage(work_arena, "user", compact_instruction)); + try std.testing.expectEqual(@as(usize, 1), dropPriorTurnReasoning(&agent)); + try std.testing.expect(trimOldestToolOutputsAlloc(&agent, work_arena) > 0); + + // The temporary request changed, while the live conversation retained both + // the active-loop reasoning item and every original output byte. + try std.testing.expectEqual(@as(usize, 12), live.items.len); + try std.testing.expectEqualStrings("reasoning", live.items[1].object.get("type").?.string); + try std.testing.expectEqual(@as(usize, 5000), live.items[2].object.get("output").?.string.len); + for (agent.messages.items) |item| { + if (item != .object) continue; + const t = item.object.get("type") orelse continue; + try std.testing.expect(!(t == .string and std.mem.eql(u8, t.string, "reasoning"))); + } + try std.testing.expect(agent.messages.items[1].object.get("output").?.string.len < 5000); +} + +test "dropPriorTurnReasoning (#174): prior-turn reasoning goes, current turn + non-responses stay" { + var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator); + defer arena_state.deinit(); + const a = arena_state.allocator(); + + var msgs = std.json.Array.init(a); + try msgs.append(try textMessage(a, "user", "turn one")); + try msgs.append(try std.json.parseFromSliceLeaky(Value, a, "{\"type\":\"reasoning\",\"encrypted_content\":\"OLD1\"}", .{})); + try msgs.append(try textMessage(a, "assistant", "reply one")); + try msgs.append(try std.json.parseFromSliceLeaky(Value, a, "{\"type\":\"reasoning\",\"encrypted_content\":\"OLD2\"}", .{})); + try msgs.append(try textMessage(a, "user", "turn two")); + try msgs.append(try std.json.parseFromSliceLeaky(Value, a, "{\"type\":\"reasoning\",\"encrypted_content\":\"CURRENT\"}", .{})); + try msgs.append(try std.json.parseFromSliceLeaky(Value, a, "{\"type\":\"function_call\",\"name\":\"bash\",\"call_id\":\"c1\",\"arguments\":\"{}\"}", .{})); + + var agent: Agent = undefined; + agent.provider = .{ .id = "codex", .kind = .responses, .auth = .bearer, .url = "", .api_key = "", .model = "gpt-5", .context = 100_000 }; + agent.messages = msgs; + + try std.testing.expectEqual(@as(usize, 2), dropPriorTurnReasoning(&agent)); + try std.testing.expectEqual(@as(usize, 5), agent.messages.items.len); + // current-turn reasoning (after the last user message) survives, in order + const kept = agent.messages.items[3].object.get("encrypted_content").?.string; + try std.testing.expectEqualStrings("CURRENT", kept); + + // Once compact() appends its synthetic user turn, the formerly-current + // reasoning belongs to a prior turn and is safe to omit from the full + // summary resend too. + try agent.messages.append(try textMessage(a, "user", compact_instruction)); + try std.testing.expectEqual(@as(usize, 1), dropPriorTurnReasoning(&agent)); + for (agent.messages.items) |m| { + if (m != .object) continue; + const t = m.object.get("type") orelse continue; + try std.testing.expect(!(t == .string and std.mem.eql(u8, t.string, "reasoning"))); + } + + // non-responses providers are untouched + agent.provider.kind = .openai; + try std.testing.expectEqual(@as(usize, 0), dropPriorTurnReasoning(&agent)); + try std.testing.expectEqual(@as(usize, 5), agent.messages.items.len); +} + +test "repeated opaque compaction transport failures unlock bounded recovery" { + var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator); + defer arena_state.deinit(); + const a = arena_state.allocator(); + var agent: Agent = undefined; + agent.provider = .{ .id = "codex", .kind = .responses, .auth = .bearer, .url = "", .api_key = "", .model = "gpt-5", .context = 100_000 }; + agent.messages = std.json.Array.init(a); + agent.sub = false; + agent.strict = false; + agent.sys_normal = ""; + agent.sys_strict = ""; + agent.tools_responses = ""; + agent.last_context_tokens = 96_000; + agent.context_local_tokens = agent.fullRequestEstimateTokens(); + agent.compact_transport_failures = 0; + agent.last_request_write_failed = true; + + try std.testing.expect(!repeatedOpaqueCompactionFailure(&agent, error.ApiError)); + try std.testing.expectEqual(@as(u8, 1), agent.compact_transport_failures); + try std.testing.expect(repeatedOpaqueCompactionFailure(&agent, error.ApiError)); + try std.testing.expectEqual(@as(u8, 2), agent.compact_transport_failures); + + // Merely crossing compact@ (80%) is not overflow proof. + agent.last_context_tokens = 90_000; + agent.compact_transport_failures = 1; + try std.testing.expect(!repeatedOpaqueCompactionFailure(&agent, error.ApiError)); + + // A non-WriteFailed provider outcome breaks the streak. + agent.last_request_write_failed = false; + try std.testing.expect(!repeatedOpaqueCompactionFailure(&agent, error.ApiError)); + try std.testing.expectEqual(@as(u8, 0), agent.compact_transport_failures); + + // Other transport failures are represented as false, not overflow evidence. + agent.compact_transport_failures = 1; + try std.testing.expect(!repeatedOpaqueCompactionFailure(&agent, error.ApiError)); + try std.testing.expectEqual(@as(u8, 0), agent.compact_transport_failures); + + // Unknown context windows never authorize destructive recovery by count. + agent.provider.context = 0; + agent.last_request_write_failed = true; + agent.compact_transport_failures = 1; + try std.testing.expect(!repeatedOpaqueCompactionFailure(&agent, error.ApiError)); +} + +test "trimOldestToolOutputs recovers a runaway tool-loop history (#163)" { + var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator); + defer arena_state.deinit(); + const a = arena_state.allocator(); + var msgs = std.json.Array.init(a); + try msgs.append(try textMessage(a, "user", "babysit the CI")); // the only clean user turn + const big = try a.alloc(u8, 5000); + @memset(big, 'x'); + var i: usize = 0; + while (i < 10) : (i += 1) { + var o: std.json.ObjectMap = .empty; + try o.put(a, "type", .{ .string = "function_call_output" }); + try o.put(a, "call_id", .{ .string = "c" }); + try o.put(a, "output", .{ .string = big }); + try msgs.append(.{ .object = o }); + } + var agent: Agent = undefined; + agent.messages = msgs; + agent.arena = a; + agent.last_context_tokens = 100000; + agent.provider = .{ .id = "codex", .kind = .responses, .auth = .bearer, .url = "", .api_key = "", .model = "gpt-5", .context = 270_000 }; + agent.sub = false; + agent.strict = false; + agent.sys_normal = ""; + agent.sys_strict = ""; + agent.tools_responses = ""; + agent.message_mutation_arena = null; + const before_request = agent.fullRequestEstimateTokens(); + agent.context_local_tokens = before_request; + // no clean user turn after the midpoint -> the old emergencyTrim would give up + try std.testing.expect(emergencyCutIndex(agent.messages.items) == null); + const reclaimed = trimOldestToolOutputs(&agent); + try std.testing.expect(reclaimed > 0); // recovered instead of wedging + // A small trim conservatively lowers, rather than replaces, the server meter. + try std.testing.expect(agent.last_context_tokens > 0); + try std.testing.expectEqual(agent.fullRequestEstimateTokens() +| (@as(u64, 100000) -| before_request), agent.last_context_tokens); + var truncated: usize = 0; + var full: usize = 0; + for (agent.messages.items) |m| { + if (m == .object) if (m.object.get("output")) |out| if (out == .string) { + if (out.string.len < 1000) truncated += 1 else full += 1; + }; + } + try std.testing.expectEqual(@as(usize, 6), truncated); // 10 outputs, oldest 6 truncated + try std.testing.expectEqual(@as(usize, 4), full); // 4 most-recent kept verbatim + try std.testing.expectEqual(@as(usize, 11), agent.messages.items.len); // no message dropped +} + +test "capOversizedToolOutputs (#193): bounds an oversized output in every wire format, leaves small ones + non-tool msgs" { + var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator); + defer arena_state.deinit(); + const a = arena_state.allocator(); + const cap: usize = 1024; + const big = try a.alloc(u8, 8192); + @memset(big, 'x'); + + var msgs = std.json.Array.init(a); + // responses function_call_output (oversized) + var fco: std.json.ObjectMap = .empty; + try fco.put(a, "type", .{ .string = "function_call_output" }); + try fco.put(a, "call_id", .{ .string = "c1" }); + try fco.put(a, "output", .{ .string = big }); + try msgs.append(.{ .object = fco }); + // openai role:"tool" (oversized) + var tool: std.json.ObjectMap = .empty; + try tool.put(a, "role", .{ .string = "tool" }); + try tool.put(a, "tool_call_id", .{ .string = "c2" }); + try tool.put(a, "content", .{ .string = big }); + try msgs.append(.{ .object = tool }); + // anthropic user turn carrying a tool_result block (oversized) + var user: std.json.ObjectMap = .empty; + try user.put(a, "role", .{ .string = "user" }); + var blocks = std.json.Array.init(a); + var tr: std.json.ObjectMap = .empty; + try tr.put(a, "type", .{ .string = "tool_result" }); + try tr.put(a, "tool_use_id", .{ .string = "c3" }); + try tr.put(a, "content", .{ .string = big }); + try blocks.append(.{ .object = tr }); + try user.put(a, "content", .{ .array = blocks }); + try msgs.append(.{ .object = user }); + // a small tool output (within cap) — must be left untouched + var small: std.json.ObjectMap = .empty; + try small.put(a, "type", .{ .string = "function_call_output" }); + try small.put(a, "output", .{ .string = "ok" }); + try msgs.append(.{ .object = small }); + // a plain assistant text message — never a tool output, left untouched + try msgs.append(try textMessage(a, "assistant", "hello")); + + var agent: Agent = undefined; + agent.arena = a; + agent.messages = msgs; + agent.last_context_tokens = 200_000; + agent.provider = .{ .id = "codex", .kind = .responses, .auth = .bearer, .url = "", .api_key = "", .model = "gpt-5", .context = 270_000 }; + agent.sub = false; + agent.strict = false; + agent.sys_normal = ""; + agent.sys_strict = ""; + agent.tools_responses = ""; + agent.message_mutation_arena = null; + // The high provider meter predates these unsent tool outputs. + agent.context_local_tokens = 8_000; + + const reclaimed = capOversizedToolOutputs(&agent, cap); + try std.testing.expect(reclaimed > 0); + // The known-underestimating local serializer must not replace a high server meter. + try std.testing.expect(agent.last_context_tokens > 0); + try std.testing.expectEqual(agent.fullRequestEstimateTokens() +| (@as(u64, 200_000) - 8_000), agent.last_context_tokens); + + // every oversized tool output is now within the cap, with a marker + const out0 = agent.messages.items[0].object.get("output").?.string; + try std.testing.expect(out0.len <= cap); + try std.testing.expect(std.mem.indexOf(u8, out0, "truncated") != null); + const out1 = agent.messages.items[1].object.get("content").?.string; + try std.testing.expect(out1.len <= cap); + const block = agent.messages.items[2].object.get("content").?.array.items[0]; + try std.testing.expect(block.object.get("content").?.string.len <= cap); + // within-cap output and the non-tool message are untouched + try std.testing.expectEqualStrings("ok", agent.messages.items[3].object.get("output").?.string); + try std.testing.expectEqualStrings("hello", agent.messages.items[4].object.get("content").?.string); + // cap == 0 disables the cap entirely (unknown window) + try std.testing.expectEqual(@as(usize, 0), capOversizedToolOutputs(&agent, 0)); +} + +test "cleanUserTurn: plain user text yes; assistant/tool_result no" { + var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator); + defer arena_state.deinit(); + const a = arena_state.allocator(); + try std.testing.expect(cleanUserTurn(try textMessage(a, "user", "hello"))); + try std.testing.expect(!cleanUserTurn(try textMessage(a, "assistant", "hi"))); + // an anthropic tool_result-only user message is NOT a clean conversation start + const tr = try std.json.parseFromSliceLeaky(Value, a, "{\"role\":\"user\",\"content\":[{\"type\":\"tool_result\",\"content\":\"ok\"}]}", .{}); + try std.testing.expect(!cleanUserTurn(tr)); +} + +test "emergencyCutIndex: cuts at a clean user turn at/after the midpoint" { + var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator); + defer arena_state.deinit(); + const a = arena_state.allocator(); + var items = std.json.Array.init(a); + const roles = [_][]const u8{ "user", "assistant", "user", "assistant", "user", "assistant", "user", "assistant" }; + for (roles) |r| try items.append(try textMessage(a, r, "x")); + try std.testing.expectEqual(@as(?usize, 4), emergencyCutIndex(items.items)); // midpoint 4 is a user turn + try std.testing.expectEqual(@as(?usize, null), emergencyCutIndex(items.items[0..3])); // too short to trim +} + +test "emergencyTrim preserves the authoritative meter after a partial cut" { + var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator); + defer arena_state.deinit(); + const a = arena_state.allocator(); + var items = std.json.Array.init(a); + const roles = [_][]const u8{ "user", "assistant", "user", "assistant", "user", "assistant", "user", "assistant" }; + for (roles) |role| try items.append(try textMessage(a, role, "a retained message with some measurable bytes")); + + var agent: Agent = undefined; + agent.arena = a; + agent.messages = items; + agent.provider = .{ .id = "codex", .kind = .responses, .auth = .bearer, .url = "", .api_key = "", .model = "gpt-5", .context = 270_000 }; + agent.last_context_tokens = 200_000; + agent.sub = false; + agent.strict = false; + agent.sys_normal = ""; + agent.sys_strict = ""; + agent.tools_responses = ""; + + const before = agent.fullInputEstimateTokens(); + const before_request = agent.fullRequestEstimateTokens(); + agent.context_local_tokens = before_request; + try std.testing.expectEqual(@as(usize, 4), emergencyTrim(&agent)); + const after = agent.fullInputEstimateTokens(); + try std.testing.expect(after < before); + const expected = agent.fullRequestEstimateTokens() +| (@as(u64, 200_000) -| before_request); + try std.testing.expectEqual(expected, agent.last_context_tokens); + try std.testing.expect(agent.last_context_tokens > 0); +} + +test "emergencyCutIndex: skips a tool_result user message at the midpoint" { + var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator); + defer arena_state.deinit(); + const a = arena_state.allocator(); + var items = std.json.Array.init(a); + try items.append(try textMessage(a, "user", "x")); // 0 + try items.append(try textMessage(a, "assistant", "x")); // 1 + try items.append(try textMessage(a, "user", "x")); // 2 + try items.append(try textMessage(a, "assistant", "x")); // 3 + // 4: an anthropic tool_result-only user message (not a valid conversation start) + try items.append(try std.json.parseFromSliceLeaky(Value, a, "{\"role\":\"user\",\"content\":[{\"type\":\"tool_result\",\"content\":\"x\"}]}", .{})); // 4 (skip) + try items.append(try textMessage(a, "assistant", "x")); // 5 + try items.append(try textMessage(a, "user", "x")); // 6 (first clean user >= midpoint) + try items.append(try textMessage(a, "assistant", "x")); // 7 + try std.testing.expectEqual(@as(?usize, 6), emergencyCutIndex(items.items)); +} diff --git a/src/agent_context.zig b/src/agent_context.zig new file mode 100644 index 00000000..92a4d20f --- /dev/null +++ b/src/agent_context.zig @@ -0,0 +1,510 @@ +//! Context-window estimation, usage accounting, and cost metering. + +const std = @import("std"); +const Io = std.Io; +const Value = std.json.Value; + +const Agent = @import("agent.zig").Agent; +const messages_mod = @import("messages.zig"); +const pricing = @import("pricing.zig"); +const g_cost = &pricing.g_cost; + +test "recordUsage (#202): floors the meter from the local estimate when usage is absent" { + var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator); + defer arena_state.deinit(); + const a = arena_state.allocator(); + + var msgs = std.json.Array.init(a); + var m: std.json.ObjectMap = .empty; + try m.put(a, "role", .{ .string = "user" }); + try m.put(a, "content", .{ .string = "the quick brown fox jumps over the lazy dog" }); + try msgs.append(.{ .object = m }); + + var agent: Agent = undefined; + agent.io = std.testing.io; + agent.arena = a; + agent.messages = msgs; + agent.last_context_tokens = 0; + agent.context_local_tokens = 0; + agent.provider = .{ .id = "anthropic", .kind = .anthropic, .auth = .x_api_key, .url = "", .api_key = "", .model = "claude", .context = 100_000 }; + agent.sub = false; + agent.strict = false; + agent.sys_normal = ""; + agent.sys_strict = ""; + agent.tools_anthropic = ""; + + // a response body with NO usage object previously froze the meter at its stale + // value; now it floors to max(full-input estimate, req_body_len/4) so the + // between-turns compaction gate can still fire. + const root: std.json.ObjectMap = .empty; + recordUsage(&agent, root, 4000); + + try std.testing.expect(agent.last_context_tokens > 0); + const expected = @max(fullRequestEstimateTokens(&agent), @as(u64, 1000)); // 4000/4 + try std.testing.expectEqual(expected, agent.last_context_tokens); + + // A fresh full-history response can authoritatively correct a stale high + // meter after an explicit history mutation. + agent.last_context_tokens = 9_000; + var usage: std.json.ObjectMap = .empty; + try usage.put(a, "input_tokens", .{ .integer = 10 }); + try usage.put(a, "output_tokens", .{ .integer = 5 }); + var response: std.json.ObjectMap = .empty; + try response.put(a, "usage", .{ .object = usage }); + recordUsage(&agent, response, 400); + try std.testing.expectEqual(@max(fullRequestEstimateTokens(&agent), @as(u64, 15)), agent.last_context_tokens); + + // A malformed smaller total must not override stronger component evidence. + try usage.put(a, "prompt_tokens", .{ .integer = 220_000 }); + try usage.put(a, "completion_tokens", .{ .integer = 1_000 }); + try usage.put(a, "total_tokens", .{ .integer = 100 }); + try response.put(a, "usage", .{ .object = usage }); + agent.provider.kind = .openai; + agent.tools_openai = ""; + recordUsage(&agent, response, 400); + try std.testing.expectEqual(@as(u64, 221_000), agent.last_context_tokens); +} + +test "fullInputEstimateTokens (#174): counts retained reasoning the chained usage never reports" { + var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator); + defer arena_state.deinit(); + const a = arena_state.allocator(); + var msgs = std.json.Array.init(a); + var agent: Agent = undefined; + agent.messages = msgs; + try std.testing.expectEqual(@as(u64, 0), fullInputEstimateTokens(&agent) / 100); // empty history ≈ nothing + // a fat encrypted-reasoning item — exactly what a WS-chained total_tokens excludes + const blob = "{\"type\":\"reasoning\",\"encrypted_content\":\"" ++ ("A" ** 8192) ++ "\"}"; + try msgs.append(try std.json.parseFromSliceLeaky(Value, a, blob, .{})); + agent.messages = msgs; + const est = fullInputEstimateTokens(&agent); + try std.testing.expect(est > 2000); // ~8KB serialized / 4 bytes-per-token +} + +test "context estimate reuses an already-serialized history byte count" { + var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator); + defer arena_state.deinit(); + const a = arena_state.allocator(); + var messages = std.json.Array.init(a); + try messages.append(try messages_mod.textMessage(a, "user", "measure this once")); + + var agent: Agent = undefined; + agent.messages = messages; + agent.provider = .{ .id = "codex", .kind = .responses, .auth = .bearer, .url = "", .api_key = "", .model = "gpt-5", .context = 100_000 }; + agent.sub = false; + agent.strict = false; + agent.sys_normal = "system"; + agent.sys_strict = "strict"; + agent.tools_responses = "[]"; + agent.last_context_tokens = 12_000; + agent.context_local_tokens = 8_000; + + const input_bytes = jsonSerializedLen(Value{ .array = messages }); + const estimate = contextEstimateFromInputBytes(&agent, input_bytes); + try std.testing.expectEqual(fullRequestEstimateTokens(&agent), estimate.local); + try std.testing.expectEqual(estimate.local + 4_000, estimate.effective); +} + +pub fn recordUsage(self: *Agent, root: std.json.ObjectMap, req_body_len: usize) void { + self.last_cache_read = 0; + self.last_usage_includes_output = false; + // #202: keep the context meter live when the provider omits usage — otherwise + // the between-turns compaction gate freezes at a stale value and a long session + // can wedge. Mirror the codex/.responses fallback (req_body_len/4, floored at the + // full-input estimate) that recordUsageResponses already applies (#174). + const usage = root.get("usage") orelse return floorContextTokens(self, req_body_len / 4); + if (usage != .object) return floorContextTokens(self, req_body_len / 4); + const u = usage.object; + switch (self.provider.kind) { + .anthropic => { + var total: i64 = 0; + const fields = [_][]const u8{ + "input_tokens", "output_tokens", + "cache_read_input_tokens", "cache_creation_input_tokens", + }; + for (fields) |f| total +|= usageInt(u, f); + if (total > 0) + replaceContextTokens(self, @intCast(total)) + else + floorContextTokens(self, req_body_len / 4); + self.last_usage_includes_output = total > 0 and if (u.get("output_tokens")) |v| v == .integer and v.integer >= 0 else false; + const cache = usageInt(u, "cache_read_input_tokens"); + if (cache > 0) self.last_cache_read = @intCast(cache); + // cache writes bill ~like input; fold them into uncached input. + self.recordCost(usageInt(u, "input_tokens") +| usageInt(u, "cache_creation_input_tokens"), cache, usageInt(u, "output_tokens")); + }, + .openai => { + const total = usageInt(u, "total_tokens"); + const computed_total = usageInt(u, "prompt_tokens") +| usageInt(u, "completion_tokens"); + const reported_total = @max(total, computed_total); + if (reported_total > 0) + replaceContextTokens(self, @intCast(reported_total)) + else + floorContextTokens(self, req_body_len / 4); + self.last_usage_includes_output = reported_total > 0 and + (total > 0 or if (u.get("completion_tokens")) |v| v == .integer and v.integer >= 0 else false); + // deepseek reports prompt_cache_hit_tokens; the OpenAI shape + // nests cached_tokens under prompt_tokens_details. + var cache = usageInt(u, "prompt_cache_hit_tokens"); + if (cache == 0) if (u.get("prompt_tokens_details")) |d| if (d == .object) { + cache = usageInt(d.object, "cached_tokens"); + }; + if (cache > 0) self.last_cache_read = @intCast(cache); + self.recordCost(@max(usageInt(u, "prompt_tokens") - cache, 0), cache, usageInt(u, "completion_tokens")); + }, + // codex uses recordUsageResponses on its own path. + .responses => {}, + } +} + +/// #202: floor the context meter at the local estimate (full-input byte/4, or the +/// request-body byte/4 when the history isn't serialized yet) when the API omits +/// usage, so auto-compaction still triggers. Never lowers an existing higher count. +fn floorContextTokens(self: *Agent, est: u64) void { + const estimate = self.contextEstimate(); + self.last_context_tokens = @max(estimate.effective, @max(estimate.local, est)); + self.context_local_tokens = estimate.local; +} + +/// A full-history request's nonzero provider usage is authoritative enough to +/// correct a stale high meter after an explicit trim/provider mutation. Still +/// retain the complete local request estimate as a structural lower bound. +fn replaceContextTokens(self: *Agent, reported: u64) void { + const local = self.fullRequestEstimateTokens(); + self.last_context_tokens = @max(local, reported); + self.context_local_tokens = local; +} + +/// An integer usage field, or 0 if absent / wrong type. +pub fn usageInt(obj: std.json.ObjectMap, name: []const u8) i64 { + if (obj.get(name)) |v| if (v == .integer and v.integer > 0) return v.integer; + return 0; +} + +/// Record one request's usage into the session-wide tally (g_cost): +/// token counts always; USD only for API-key providers with a +/// price_table row. Subscription providers (codex, claude) bill flat +/// and tally as sub_calls; unpriced models as unpriced_calls. +pub fn recordCost(self: *Agent, uncached_in: i64, cache_in: i64, out: i64) void { + g_cost.add(self.io, self.provider.id, self.provider.model, uncached_in, cache_in, out); +} + +/// Count a value's serialized JSON bytes without allocating the output. +fn jsonSerializedLen(value: anytype) usize { + var buf: [512]u8 = undefined; + var d: Io.Writer.Discarding = .init(&buf); + var s: std.json.Stringify = .{ .writer = &d.writer }; + // Discarding is infallible today. Preserve the bytes counted so far if a + // future writer becomes stricter instead of returning a spurious zero. + s.write(value) catch return d.fullCount(); + return d.fullCount(); +} + +/// #174: ~4-bytes/token estimate of the FULL history serialized as Responses +/// `input` items — the cost of the next full-history resend (runTurn closes +/// the WS per turn, so every turn's first request replays everything). The +/// chained WS usage can sit far below this: with previous_response_id the +/// server discards prior-turn reasoning from context, while a resend pays for +/// every retained encrypted reasoning item again. Counting discard writer — +/// no allocation. +pub fn fullInputEstimateTokens(self: *Agent) u64 { + return @intCast(jsonSerializedLen(Value{ .array = self.messages }) / 4); +} + +/// Estimate the complete fixed+history input sent by a normal model request. +/// fullInputEstimateTokens intentionally covers messages only; the pre-send gate +/// must additionally count the mutable system prompt and active tool schema. Keep +/// the old 8k conservative baseline when those serialize smaller, but never let it +/// hide an unbounded set_system_prompt/set_agent payload. +fn requestEstimateFromInputBytes(self: *Agent, input_bytes: usize) u64 { + const system_bytes = jsonSerializedLen(self.systemPrompt()); + const total_bytes = input_bytes +| system_bytes +| self.toolsJson().len; + const measured: u64 = @intCast(total_bytes / 4); + const input_tokens: u64 = @intCast(input_bytes / 4); + const baseline = @min(@as(u64, 8000), self.provider.context / 8); + return @max(measured, input_tokens +| baseline); +} + +pub fn fullRequestEstimateTokens(self: *Agent) u64 { + return requestEstimateFromInputBytes(self, jsonSerializedLen(Value{ .array = self.messages })); +} + +pub const ContextEstimate = struct { + local: u64, + effective: u64, +}; + +/// Compute the locally measurable request size and the hidden-token-adjusted +/// occupancy in one history serialization. Callers that need both previously +/// walked the entire JSON history twice. +pub fn contextEstimate(self: *Agent) ContextEstimate { + return self.contextEstimateFromInputBytes(jsonSerializedLen(Value{ .array = self.messages })); +} + +/// Variant for callers already serializing the history (session persistence). +/// Reusing the observed byte count avoids a redundant full JSON walk. +pub fn contextEstimateFromInputBytes(self: *Agent, input_bytes: usize) ContextEstimate { + const local = requestEstimateFromInputBytes(self, input_bytes); + return .{ + .local = local, + .effective = local +| (self.last_context_tokens -| self.context_local_tokens), + }; +} + +/// Current context occupancy: today's locally measurable request plus the +/// server-only delta paired with the last usage sample. This automatically +/// counts user/tool content appended after that sample without double-counting +/// provider output once step* advances the local anchor. +pub fn effectiveContextTokens(self: *Agent) u64 { + return self.contextEstimate().effective; +} + +/// Pair a fresh provider usage sample with history after its response items +/// have been appended. Those items are already represented in total_tokens; +/// advancing only the local anchor prevents counting them twice. +pub fn pairContextMeterWithCurrentLocal(self: *Agent) void { + if (!self.last_usage_includes_output) return; + const local = self.fullRequestEstimateTokens(); + self.last_context_tokens = @max(self.last_context_tokens, local); + self.context_local_tokens = local; + self.last_usage_includes_output = false; +} + +/// Materialize local input mutations while retaining the server-only delta. +pub fn rebaseContextMeter(self: *Agent) void { + const estimate = self.contextEstimate(); + self.last_context_tokens = estimate.effective; + self.context_local_tokens = estimate.local; + self.last_cache_read = 0; +} + +test "rebaseContextMeter preserves hidden context while replacing local input" { + var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator); + defer arena_state.deinit(); + const a = arena_state.allocator(); + var agent: Agent = undefined; + agent.messages = std.json.Array.init(a); + agent.provider = .{ .id = "codex", .kind = .responses, .auth = .bearer, .url = "", .api_key = "", .model = "gpt-5", .context = 100_000 }; + agent.sub = false; + agent.strict = false; + agent.sys_normal = "x" ** 40_000; + agent.sys_strict = ""; + agent.tools_responses = ""; + agent.last_cache_read = 99; + const old_local = agent.fullRequestEstimateTokens(); + agent.last_context_tokens = old_local + 12_345; + agent.context_local_tokens = old_local; + + agent.sys_normal = "short"; + rebaseContextMeter(&agent); + try std.testing.expectEqual(agent.fullRequestEstimateTokens() + 12_345, agent.last_context_tokens); + try std.testing.expectEqual(@as(u64, 0), agent.last_cache_read); +} + +test "context anchor counts unsampled growth and pairs provider-covered output" { + var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator); + defer arena_state.deinit(); + const a = arena_state.allocator(); + var agent: Agent = undefined; + agent.messages = std.json.Array.init(a); + agent.provider = .{ .id = "codex", .kind = .responses, .auth = .bearer, .url = "", .api_key = "", .model = "gpt-5", .context = 100_000 }; + agent.sub = false; + agent.strict = false; + agent.sys_normal = ""; + agent.sys_strict = ""; + agent.tools_responses = ""; + const base_local = agent.fullRequestEstimateTokens(); + agent.last_context_tokens = 50_000; + agent.context_local_tokens = base_local; + agent.last_usage_includes_output = false; + + try agent.messages.append(try messages_mod.textMessage(a, "assistant", "x" ** 4000)); + agent.pairContextMeterWithCurrentLocal(); + try std.testing.expect(agent.effectiveContextTokens() > 50_000); + try std.testing.expectEqual(base_local, agent.context_local_tokens); + + agent.last_context_tokens = 60_000; + agent.last_usage_includes_output = true; + agent.pairContextMeterWithCurrentLocal(); + try std.testing.expectEqual(agent.fullRequestEstimateTokens(), agent.context_local_tokens); + try std.testing.expectEqual(@max(@as(u64, 60_000), agent.fullRequestEstimateTokens()), agent.effectiveContextTokens()); +} + +/// Pre-send overflow gate (#193). `last_context_tokens` only updates from the +/// server's returned usage, so it lags tool output appended *within* a turn: a +/// burst of large tool results can push this turn's input past the model's wall +/// before the between-turns 80% meter ever sees it (the "input exceeds the +/// context window" rejection on codex/gpt-5.x). Estimating the full input +/// locally before each request lets the turn loop compact first — the analogue +/// of codex's run_pre_sampling_compact / opencode's isOverflow-before-each-turn. +/// Guarded on a known window (compactAt()==0 → don't compact blindly). +pub fn inputOverCompactThreshold(self: *Agent) bool { + const threshold = self.provider.compactAt(); + if (threshold == 0) return false; + // The server-reported meter (the same last_context_tokens the between-turns + // gates at mainloop.zig:683/734 already trust) is refreshed mid-turn by + // recordUsageResponses (agent_request.zig:905/924). On codex/.responses it can + // sit far above the local byte/4 estimate — retained encrypted-reasoning items + // are billed server-side but the local serialize under-counts them — so on a + // long turn it held ~511k while fullInputEstimateTokens stayed under the wall + // and the next resend was rejected for "input exceeds the context window" + // before any compaction ran. Trip a pre-send compact on the accurate meter too. + return self.effectiveContextTokens() >= threshold; +} + +test "inputOverCompactThreshold (#193): local estimate gates a pre-send compact" { + var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator); + defer arena_state.deinit(); + const a = arena_state.allocator(); + var msgs = std.json.Array.init(a); + var agent: Agent = undefined; + agent.messages = msgs; + agent.last_context_tokens = 0; // the gate now reads this too; `= undefined` won't apply the field default + agent.context_local_tokens = 0; + agent.sub = false; + agent.strict = false; + agent.sys_normal = ""; + agent.sys_strict = ""; + agent.tools_responses = ""; + // small window: compactAt() = 10_000/10*8 = 8_000 tokens ≈ 32_000 serialized bytes + agent.provider = .{ .id = "codex", .kind = .responses, .auth = .bearer, .url = "", .api_key = "", .model = "gpt-5", .context = 10_000 }; + try std.testing.expect(!inputOverCompactThreshold(&agent)); // empty history → under + // one fat tool output (~40KB serialized ≈ 10k est tokens) crosses 8k in a single append + const big = "{\"type\":\"function_call_output\",\"output\":\"" ++ ("x" ** 40000) ++ "\"}"; + try msgs.append(try std.json.parseFromSliceLeaky(Value, a, big, .{})); + agent.messages = msgs; + try std.testing.expect(inputOverCompactThreshold(&agent)); // burst → over → compact before send + // the server-reported meter alone trips the gate even with an empty local history — + // the codex mid-turn case where retained reasoning items undercount the byte/4 estimate + agent.messages = std.json.Array.init(a); + agent.last_context_tokens = agent.provider.compactAt(); + agent.context_local_tokens = agent.fullRequestEstimateTokens(); + try std.testing.expect(inputOverCompactThreshold(&agent)); + agent.last_context_tokens = 0; + agent.context_local_tokens = 0; + // Mutable system prompts are real input, not a fixed 8k prefill. A large + // prompt must trip the gate even with no message history. + agent.sys_normal = "x" ** 40_000; + try std.testing.expect(inputOverCompactThreshold(&agent)); + agent.sys_normal = ""; + // unknown window (context 0) never gates + agent.provider.context = 0; + try std.testing.expect(!inputOverCompactThreshold(&agent)); +} + +pub fn recordUsageResponses(self: *Agent, response: std.json.ObjectMap, req_body_len: usize) void { + self.last_cache_read = 0; + self.last_usage_includes_output = false; + // Fallback estimate (~4 bytes/token) from the serialized request body, + // which holds the full conversation — keeps the ctx meter and the + // compact@ trigger live when codex omits usage counts entirely. + const est: u64 = req_body_len / 4; + // A held Codex WS sends only previous_response_id + the unsent delta, so + // body/4 can be tiny even when the preceding response reported a near-full + // context. Missing or malformed usage is not evidence that the context got + // smaller: preserve the authoritative meter and floor it with every local + // estimate, just like recordUsage does for the other provider shapes. + const usage = response.get("usage") orelse return floorContextTokens(self, est); + if (usage != .object) return floorContextTokens(self, est); + const u = usage.object; + const in_tokens = usageInt(u, "input_tokens"); + const out_tokens = usageInt(u, "output_tokens"); + const raw_total_tokens = usageInt(u, "total_tokens"); + const total_tokens = @max(raw_total_tokens, in_tokens +| out_tokens); + if (total_tokens > 0) { + if (self.codex_prev_id != null) + floorContextTokens(self, @intCast(total_tokens)) + else + replaceContextTokens(self, @intCast(total_tokens)); + } else floorContextTokens(self, est); + const usage_covers_output = total_tokens > 0 and + (raw_total_tokens > 0 or if (u.get("output_tokens")) |v| v == .integer and v.integer >= 0 else false); + // Chained WS totals are not comparable to the conservative full-resend + // meter, so they cannot prove how much of the appended response growth is + // covered. Never advance that anchor; conservative double-counting is + // corrected by the next full re-anchor and cannot miss compaction. + self.last_usage_includes_output = usage_covers_output and self.codex_prev_id == null; + // #174: the server's chained number undercounts what the next full-history + // resend will cost, and the resend is the request that gets rejected — so + // the meter must never sit below the local full-input estimate. Same + // correction codex CLI applies (get_non_last_reasoning_items_tokens). + // Without it a long Extra-high session reads "95k/270k" right up until the + // backend rejects the resend for exceeding the window, and auto-compaction + // (gated on this meter) never rescues it. + var cached: i64 = 0; + if (u.get("input_tokens_details")) |d| if (d == .object) { + cached = usageInt(d.object, "cached_tokens"); + if (cached > 0) self.last_cache_read = @intCast(cached); + }; + self.recordCost(@max(in_tokens - cached, 0), cached, out_tokens); +} + +test "recordUsageResponses: usage fallbacks never lower an authoritative meter" { + var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator); + defer arena_state.deinit(); + const a = arena_state.allocator(); + + var agent: Agent = undefined; + agent.io = std.testing.io; + agent.messages = std.json.Array.init(a); + agent.provider = .{ .id = "codex", .kind = .responses, .auth = .bearer, .url = "", .api_key = "", .model = "gpt-5", .context = 10_000 }; + agent.last_cache_read = 0; + agent.last_context_tokens = 9_000; // above compactAt(), while the WS delta below is tiny + agent.codex_prev_id = null; + agent.sub = false; + agent.strict = false; + agent.sys_normal = ""; + agent.sys_strict = ""; + agent.tools_responses = ""; + agent.context_local_tokens = agent.fullRequestEstimateTokens(); + + const missing: std.json.ObjectMap = .empty; + recordUsageResponses(&agent, missing, 400); + try std.testing.expectEqual(@as(u64, 9_000), agent.last_context_tokens); + try std.testing.expect(!agent.last_usage_includes_output); + try std.testing.expect(inputOverCompactThreshold(&agent)); + + var malformed: std.json.ObjectMap = .empty; + try malformed.put(a, "usage", .{ .string = "unknown" }); + recordUsageResponses(&agent, malformed, 400); + try std.testing.expectEqual(@as(u64, 9_000), agent.last_context_tokens); + + const empty_usage: std.json.ObjectMap = .empty; + var empty_response: std.json.ObjectMap = .empty; + try empty_response.put(a, "usage", .{ .object = empty_usage }); + recordUsageResponses(&agent, empty_response, 400); + try std.testing.expectEqual(@as(u64, 9_000), agent.last_context_tokens); + + var smaller_usage: std.json.ObjectMap = .empty; + try smaller_usage.put(a, "input_tokens", .{ .integer = 90 }); + try smaller_usage.put(a, "output_tokens", .{ .integer = 10 }); + try smaller_usage.put(a, "total_tokens", .{ .integer = 100 }); + var smaller_response: std.json.ObjectMap = .empty; + try smaller_response.put(a, "usage", .{ .object = smaller_usage }); + // A full re-anchor trusts its provider reading (with the local floor). + recordUsageResponses(&agent, smaller_response, 400); + try std.testing.expectEqual(@max(fullRequestEstimateTokens(&agent), @as(u64, 100)), agent.last_context_tokens); + try std.testing.expect(agent.last_usage_includes_output); + + // The aggregate is advisory when larger input/output components are + // present; never lower the meter below the stronger provider evidence. + try smaller_usage.put(a, "input_tokens", .{ .integer = 8_000 }); + try smaller_usage.put(a, "output_tokens", .{ .integer = 1_000 }); + try smaller_response.put(a, "usage", .{ .object = smaller_usage }); + agent.last_context_tokens = 0; + recordUsageResponses(&agent, smaller_response, 400); + try std.testing.expectEqual(@max(fullRequestEstimateTokens(&agent), @as(u64, 9_000)), agent.last_context_tokens); + + // A held WS delta cannot prove the full chained context shrank. + agent.last_context_tokens = 9_000; + agent.codex_prev_id = "resp_previous"; + recordUsageResponses(&agent, smaller_response, 400); + try std.testing.expectEqual(@as(u64, 9_000), agent.last_context_tokens); + try std.testing.expect(!agent.last_usage_includes_output); + + // A fresh meter still advances from the fallback estimate. + agent.codex_prev_id = null; + agent.last_context_tokens = 0; + recordUsageResponses(&agent, missing, 4_000); + try std.testing.expectEqual(@max(fullRequestEstimateTokens(&agent), @as(u64, 1_000)), agent.last_context_tokens); +} diff --git a/src/agent_eval.zig b/src/agent_eval.zig new file mode 100644 index 00000000..f853fc2f --- /dev/null +++ b/src/agent_eval.zig @@ -0,0 +1,211 @@ +//! Eval-driven scoring loop and its optional LLM-as-judge scorer. + +const std = @import("std"); +const Io = std.Io; + +const Agent = @import("agent.zig").Agent; +const ExecResult = @import("tools.zig").ExecResult; +const ToolCtx = @import("tools.zig").ToolCtx; +const ToolOutput = @import("tools.zig").ToolOutput; +const parseEvalScore = @import("repl_glue.zig").parseEvalScore; +const utf8Prefix = @import("util.zig").utf8Prefix; + +const scoring = @import("scoring.zig"); +const promptFingerprint = scoring.promptFingerprint; +const providerClass = scoring.providerClass; +const signScore = scoring.signScore; + +const telemetry = @import("telemetry.zig"); +const runCapped = @import("jobs.zig").runCapped; +const judgeTask = @import("subagent.zig").judgeTask; + +/// Run the configured --eval scoring command, append the result to the +/// scores log (.graff/eval-log.tsv), and return a verdict for the model: +/// score (0-100), best so far, target, and whether the target is met. The +/// harness runs the command, so the model cannot fake the number. (eval tool) +pub fn runEval(self: *Agent, note: []const u8) !ExecResult { + const cmd = self.eval_cmd orelse return .{ + .text = "no eval command configured - relaunch graff with --eval and --until , or ask the user to set one", + .is_error = true, + }; + const run = runCapped(self.gpa, self.io, &.{ "/bin/sh", "-c", cmd }, 64 * 1024, 16 * 1024, 0) catch |e| + return .{ .text = try std.fmt.allocPrint(self.arena, "eval command could not run: {t}", .{e}), .is_error = true }; + defer self.gpa.free(run.stdout); + defer self.gpa.free(run.stderr); + self.eval_iter += 1; + const exit_code: i32 = switch (run.term) { + .exited => |c| @intCast(c), + else => -1, + }; + const det = parseEvalScore(run.stdout) orelse parseEvalScore(run.stderr); + + // LLM-as-judge (--judge): an independent subagent inspects the actual + // artifacts against the rubric and returns its own 0-100 score. Both + // scores must clear the target, so the binding value is their min(). + // Skip the judge when the deterministic command itself produced no + // score - fix that first rather than burn a judge run. + const judge: ?f64 = if (self.eval_judge != null and det != null) self.runJudge(self.eval_judge.?, run.stdout, note) else null; + const combined: ?f64 = if (self.eval_judge == null) + det + else if (det != null and judge != null) + @min(det.?, judge.?) + else + null; + + const target_f: f64 = @floatFromInt(self.eval_target); + const improved = if (combined) |s| (self.eval_best < 0 or s > self.eval_best) else false; + if (combined) |s| { + if (self.eval_best < 0 or s > self.eval_best) self.eval_best = s; + } + const met = if (combined) |s| s >= target_f else false; + self.appendEvalLog(note, det, judge, combined, exit_code, met) catch {}; + + // Feed the eval-driven score into the fleet (docs/hyperagents.md §9.B): + // on a NEW BEST, submit the genome (this agent's persona) with its achieved + // score on the pinned eval set (the eval command's fingerprint). Without this + // the score only ever reached .graff/eval-log.tsv and the DGM/fleet never saw + // real eval-driven work — only darwincode/JSON-proto runs ever submitted. + if (combined) |s| { + if (improved and s > 0) { // s>0: skip the initial-state / total-failure 0 (don't pollute the cell mean) + if (s > 100) { + // Review F8: parseEvalScore does NOT bound its result (a stray + // "score: 9000" line parses as 9000) — an out-of-[0,100] score + // must never be signed or submitted. Skip the score AND its + // paired propose/submit, mirroring mainloop /score's explicit + // rejection, so the submit counter stays in sync with stored + // scores. The local eval verdict below still shows the number. + if (self.tracer) |tr| tr.note("fleet", "score skipped: eval score outside [0,100]"); + } else if (telemetry.g_telem) |t| { + const sys = self.systemPrompt(); + const genome_fp = promptFingerprint(sys); + const esh_fp = promptFingerprint(cmd); + const genome: []const u8 = &genome_fp; + const esh: []const u8 = &esh_fp; + const run_id: []const u8 = &scoring.g_run_id; + const pclass = providerClass(self.provider.model); + // --niche tags this score's cell. Without it the score lands in the + // anonymous "" niche, which pullElites can never match to a builtin — + // so an eval session that wants to grow a champion must name its role. + // Truncated to 64 chars and sanitized (tab/newline/CR → ' ', review + // F7) BEFORE signing (fleetEvent's own niche cap, same as mainloop + // /score) so signed bytes equal ingested bytes. + var niche_buf: [64]u8 = undefined; + const niche = scoring.sanitizeMetaField(&niche_buf, utf8Prefix(self.eval_niche, 64)); + // Genome-send (graff-dgm.md §B): the eval genome is this agent's own + // persona, never spawned via runSub, so its prompt_text never reached + // the worker. A cell only promotes when harness_scores joins to a + // harness_genomes row, so ride the genome text over on a `propose` + // (deduped by prompt_sha) before the score — else a winning eval cell + // has nothing to serve. Gated on a niche: a "" cell is unpromotable. + // Oversized genomes skip the propose (review F6): the server verifies + // the fingerprint over the carried text, so a truncated genome would + // be dropped there anyway. + if (niche.len > 0) { + if (sys.len <= telemetry.Telemetry.max_propose_text) + t.fleetEvent("propose", niche, genome, "", pclass, "", 0, sys) + else if (self.tracer) |tr| tr.note("fleet", "propose skipped: genome > 64KB"); + } + // SCORE SCALE CONTRACT (issue #168 Gap 4): local UX stays + // 0-100 (the /100 verdicts below, eval_best, eval_target), + // but every score that leaves the client is [0,1] — divide at + // the emission boundary; s01 is what gets signed (v2: niche + + // provider_class in the envelope) and sent. + const s01 = s / 100.0; + const sig = signScore(genome, "", s01, run_id, "", "", esh, niche, pclass); + const sig_s: []const u8 = if (scoring.g_score_key != null) &sig else ""; + var provbuf: [512]u8 = undefined; + const prov = std.fmt.bufPrint(&provbuf, "{s}\t{s}\t{s}\t{s}\t{s}", .{ "", "", esh, pclass, niche }) catch ""; + t.scoreEvent(genome, "", s01, run_id, sig_s, prov); + t.fleetEvent("submit", niche, genome, "", pclass, esh, 0, ""); + } + } + } + + var aw: Io.Writer.Allocating = .init(self.arena); + const w = &aw.writer; + if (combined) |s| { + if (self.eval_judge != null) { + try w.print("eval #{d}: deterministic {d:.1} + judge {d:.1} -> {d:.1}/100 (best {d:.1}, target {d}). ", .{ self.eval_iter, det.?, judge.?, s, self.eval_best, self.eval_target }); + } else { + try w.print("eval #{d}: score {d:.1}/100 (best {d:.1}, target {d}). ", .{ self.eval_iter, s, self.eval_best, self.eval_target }); + } + if (met) + try w.writeAll("TARGET MET - finish and report the final scores.") + else if (improved) + try w.writeAll("Improved - keep going: fix the next biggest failure with one focused change.") + else + try w.writeAll("No gain over the best - try a different change; do not build on a regression."); + } else if (self.eval_judge != null and det != null and judge == null) { + try w.print("eval #{d}: deterministic score {d:.1}/100, but the judge returned no parseable score (it may have errored). Re-run after checking the rubric. ", .{ self.eval_iter, det.? }); + } else { + try w.print("eval #{d}: command ran but no score parsed - print a bare number, or JSON with a score field (0-100 or 0-1), on the last line. ", .{self.eval_iter}); + } + const tail = if (run.stdout.len > 1500) run.stdout[run.stdout.len - 1500 ..] else run.stdout; + try w.print("\n[exit {d}] eval output (tail):\n{s}", .{ exit_code, tail }); + return .{ .text = try self.arena.dupe(u8, aw.writer.buffered()), .is_error = false }; +} + +/// Append one tab-separated row to the scores log (.graff/eval-log.tsv). +/// Best-effort - a failed write never breaks the loop. +pub fn appendEvalLog(self: *Agent, note: []const u8, det: ?f64, judge: ?f64, score: ?f64, exit_code: i32, met: bool) !void { + Io.Dir.cwd().createDir(self.io, ".graff", .default_dir) catch {}; + const path = ".graff/eval-log.tsv"; + const existing = Io.Dir.cwd().readFileAlloc(self.io, path, self.arena, .limited(2 * 1024 * 1024)) catch ""; + var aw: Io.Writer.Allocating = .init(self.arena); + const w = &aw.writer; + try w.writeAll(existing); + if (existing.len > 0 and existing[existing.len - 1] != '\n') try w.writeByte('\n'); + try w.print("iter={d}\tscore=", .{self.eval_iter}); + if (score) |s| try w.print("{d:.2}", .{s}) else try w.writeAll("NA"); + try w.writeAll("\tdet="); + if (det) |d| try w.print("{d:.2}", .{d}) else try w.writeAll("NA"); + try w.writeAll("\tjudge="); + if (judge) |j| try w.print("{d:.2}", .{j}) else try w.writeAll("NA"); + try w.print("\tbest={d:.2}\ttarget={d}\tmet={s}\texit={d}\tnote=", .{ self.eval_best, self.eval_target, if (met) "yes" else "no", exit_code }); + for (note) |ch| try w.writeByte(if (ch < 0x20) ' ' else ch); + try w.writeByte('\n'); + Io.Dir.cwd().writeFile(self.io, .{ .sub_path = path, .data = aw.writer.buffered() }) catch {}; +} + +/// Spawn an independent LLM judge (a read-only subagent) to score the +/// current work against the --judge rubric on a 0-100 scale. The judge +/// inspects the real artifacts with its own tools and ends its report with +/// a `score:` line, parsed the same way as a deterministic eval. Runs on a +/// pool thread via judgeTask (mirrors workflowTask) so the eval handler can +/// await it. Returns null if the judge could not run or gave no score. +pub fn runJudge(self: *Agent, rubric: []const u8, eval_output: []const u8, note: []const u8) ?f64 { + const ctx: ToolCtx = .{ + .gpa = self.gpa, + .io = self.io, + .client = self.client, + .provider = self.provider, + .registry = if (self.sub) null else self.registry, + .from_sub = self.sub, + .approvals = self.approvals, + .tracer = self.tracer, + .snapshots = self.snapshots, + .tools_used = &self.tools_used, + }; + const evidence = if (eval_output.len > 1200) eval_output[eval_output.len - 1200 ..] else eval_output; + const what = if (note.len > 0) note else "(no note given)"; + const judge_prompt = std.fmt.allocPrint(self.arena, + \\Score the current state of the work in this directory against the rubric below, on a 0-100 scale. + \\ + \\RUBRIC: + \\{s} + \\ + \\The author's note on the latest change: {s} + \\ + \\An automated check was also run; its output (tail) is below as evidence. Form your OWN independent judgement of how well the artifacts satisfy the rubric - do not simply echo the check: + \\--- + \\{s} + \\--- + \\ + \\Inspect the actual files and artifacts the work produced (read them with your tools; do not modify anything), then score how fully they satisfy the rubric. End your reply with a single final line `score: ` where N is an integer from 0 to 100. + , .{ rubric, what, evidence }) catch return null; + var fut: Io.Future(ToolOutput) = self.io.async(judgeTask, .{ ctx, judge_prompt }); + const out = fut.await(self.io); + defer self.gpa.free(out.text); + if (out.is_error) return null; + return parseEvalScore(out.text); +} diff --git a/src/agent_request.zig b/src/agent_request.zig index a2adf266..61e3d736 100644 --- a/src/agent_request.zig +++ b/src/agent_request.zig @@ -1,36 +1,22 @@ -//! The provider round trip: request() builds+sends the body (with the -//! retry/backoff loop for flaky transport and 429/5xx), buildBody() -//! serializes it per wire format (Anthropic/OpenAI/Responses), and -//! recordUsage/recordUsageResponses/recordCost tally tokens + cost into -//! the session-wide g_cost. parseResponses reassembles the Codex/Responses -//! SSE stream into a synthetic {output, usage} object (the streaming -//! Anthropic/OpenAI SSE reassembly — assembleAnthropic/assembleOpenAI — -//! lives in agent_steps.zig instead, since it feeds the step* functions -//! there). Split out of the Agent struct (#123, 600-line goal). +//! Provider round trip and retry loop. +//! +//! Request policy, context accounting, Responses reassembly, and body +//! serialization live in focused sibling modules and are re-exported here so +//! Agent's public method aliases remain source-compatible. const std = @import("std"); const Io = std.Io; const Value = std.json.Value; const main_mod = @import("main.zig"); -const agent_mod = @import("agent.zig"); -const Agent = agent_mod.Agent; -const max_tokens = main_mod.max_tokens; - -const pricing = @import("pricing.zig"); +const Agent = @import("agent.zig").Agent; const oauth = @import("oauth.zig"); -const g_cost = &pricing.g_cost; const messages_mod = @import("messages.zig"); const sanitizeMessagesUtf8 = messages_mod.sanitizeMessagesUtf8; const normalizeResponsesHistory = messages_mod.normalizeResponsesHistory; const normalizeOpenAIHistory = messages_mod.normalizeOpenAIHistory; -const serde = @import("serde.zig"); -const writeAnthropicMessages = serde.writeAnthropicMessages; -const writeOpenAIMessageNormalized = serde.writeOpenAIMessageNormalized; -const util = @import("util.zig"); - const http = @import("http.zig"); const postWatched = http.postWatched; const RetryPlan = http.RetryPlan; @@ -38,292 +24,59 @@ const RetryPlan = http.RetryPlan; const tools_mod = @import("tools.zig"); const apiErrorMessage = tools_mod.apiErrorMessage; const mentionsReasoningEffort = tools_mod.mentionsReasoningEffort; - const telemetry = @import("telemetry.zig"); -/// POST the current history; returns the parsed response root object -/// (arena-owned). Reports API error envelopes and returns error.ApiError. -/// In strict mode we force tool_choice; if a provider rejects that (e.g. -/// the codegraff gateway with thinking on), we retry once without forcing -/// and lean on the strict system prompt instead. Root requests stream: -/// text deltas print live (postStream) and the buffered SSE events are -/// reassembled into the non-streaming response shape afterwards. -/// #148: does a provider error message look like an auth failure (a stale/ -/// revoked OAuth token), so we refresh + retry rather than give up? Matches the -/// common 401 phrasings ("invalid api key", "expired", "unauthorized", …). -fn isAuthError(msg: []const u8) bool { - return std.ascii.indexOfIgnoreCase(msg, "api key") != null or - std.ascii.indexOfIgnoreCase(msg, "unauthorized") != null or - std.ascii.indexOfIgnoreCase(msg, "expired") != null or - std.ascii.indexOfIgnoreCase(msg, "authentication") != null or - std.ascii.indexOfIgnoreCase(msg, "invalid_api_key") != null; -} - -test "isAuthError (#148): auth failures only, not credits/rate/other" { - // real provider 401 phrasings → refresh + retry - try std.testing.expect(isAuthError("The API Key appears to be invalid or may have expired.")); - try std.testing.expect(isAuthError("Unauthorized")); - try std.testing.expect(isAuthError("authentication_error")); - try std.testing.expect(isAuthError("invalid_api_key")); - // NOT auth — must never trigger a refresh loop - try std.testing.expect(!isAuthError("You have run out of credits or need a Grok subscription.")); - try std.testing.expect(!isAuthError("rate limit exceeded")); - try std.testing.expect(!isAuthError("model not found")); - try std.testing.expect(!isAuthError("context length exceeded")); -} - -/// True if `msg` is a provider's "input is over the context window" rejection, -/// across wire formats: codex/responses ("exceeds the context window"), openai -/// ("maximum context length", "context_length_exceeded"), anthropic ("prompt is -/// too long", "exceed context limit"). Drives the in-turn emergency-trim + retry -/// recovery symmetrically for every provider (#193) — before it, only the codex -/// path recovered and anthropic/openai died on an over-window turn. -fn isContextOverflow(msg: []const u8, code: ?[]const u8) bool { - // #203: match the structured error code first (openai/codex parity — a local or - // non-English provider whose message text differs still recovers), then fall back - // to the human-readable phrasing. - if (code) |c| { - const codes = [_][]const u8{ "context_length_exceeded", "context_window_exceeded" }; - for (codes) |k| if (std.mem.eql(u8, c, k)) return true; - } - const needles = [_][]const u8{ - "context window", // codex/responses: "exceeds the context window" - "context length", // openai: "maximum context length is N tokens" - "context_length_exceeded", // openai error code echoed into the message - "context limit", // anthropic: "input length and max_tokens exceed context limit" - "prompt is too long", // anthropic: "prompt is too long: N tokens > M maximum" - "maximum context", // defensive: "maximum context ... exceeded" - }; - for (needles) |n| if (std.mem.indexOf(u8, msg, n) != null) return true; - return false; -} - -/// The structured error code from a parsed error envelope, if any: openai / lmstudio / -/// deepseek put it at root.error.code; some providers use a top-level root.code (#203). -fn errorCode(root: std.json.ObjectMap) ?[]const u8 { - if (root.get("error")) |ev| { - if (ev == .object) { - if (ev.object.get("code")) |cv| { - if (cv == .string) return cv.string; - } - } - } - if (root.get("code")) |cv| { - if (cv == .string) return cv.string; - } - return null; -} - -/// #193 follow-up: shared in-turn context-overflow recovery for the three -/// anthropic/openai error branches (streamed error event, non-streamed -/// `{"type":"error"}` envelope, and the generic apiErrorMessage path). Before -/// this, only the codex/.responses branch recovered — anthropic/openai died on an -/// over-window turn. Returns true if the caller should `continue` the rebuild loop -/// (emergency-trimmed, retry the same request once); false to fall through to the -/// normal error. Pins the meter to the window FIRST so the between-turns -/// compaction engages even when we can't recover here (the rejected request -/// returns no usage to correct the lagging meter). Guarded by `retried` (one -/// `context_retried` shared across every branch of a request) so a second overflow -/// falls through and never loops. These wire formats send the full input each -/// rebuild, so — unlike the codex branch — no closeCodexWs re-anchor is needed. -fn recoverContextOverflow(self: *Agent, msg: []const u8, code: ?[]const u8, retried: *bool) bool { - if (!isContextOverflow(msg, code)) return false; - self.last_context_tokens = self.provider.context; - if (retried.* or self.emergencyTrim() == 0) return false; - retried.* = true; - if (self.tracer) |tr| tr.note("context", "input over the window — emergency-trimmed and retrying the turn"); - return true; -} - -const max_server_retries: usize = 3; // #opencode-parity: bounded retries for a transient in-stream server overload - -/// #opencode-parity: an in-band error event (an SSE {"type":"error"} or a JSON -/// error envelope) naming a TRANSIENT server condition — Anthropic overloaded_error, -/// OpenAI server_error / server_is_overloaded, or plain "overloaded" — is a 5xx that -/// surfaced mid-stream and should be retried, not hard-failed. Billing / quota / -/// invalid-input errors are NOT transient and fall through to a hard fail. -fn isTransientServerError(etype: []const u8, code: ?[]const u8, msg: []const u8) bool { - const needles = [_][]const u8{ "overloaded", "server_error", "server_is_overloaded" }; - for (needles) |n| { - if (std.ascii.indexOfIgnoreCase(etype, n) != null) return true; - if (std.ascii.indexOfIgnoreCase(msg, n) != null) return true; - if (code) |c| if (std.ascii.indexOfIgnoreCase(c, n) != null) return true; - } - return false; -} - -/// If an in-stream error names a transient server overload, back off and retry the -/// request (bounded), like a 5xx — returns true to signal the caller to `continue`. -/// Esc during the backoff propagates as error.Interrupted. #opencode-parity. -fn retryTransientServerError(self: *Agent, etype: []const u8, code: ?[]const u8, msg: []const u8, retries: *usize) !bool { - if (!isTransientServerError(etype, code, msg)) return false; - if (retries.* >= max_server_retries) return false; - retries.* += 1; - self.partial_text.clearRetainingCapacity(); // fresh re-stream after the retry, no concat - const delay_ms = RetryPlan.delayMs(true, retries.* - 1); // 1·2·4s - try self.say("[server overloaded — retrying in {d}s ({d}/{d})]\n", .{ delay_ms / 1000, retries.*, max_server_retries }); - if (self.tracer) |tr| tr.note("retry", "server overloaded (in-stream)"); - if (telemetry.g_telem) |t| t.errorEvent("server_overloaded", if (msg.len > 0) msg else etype); - self.sleepInterruptible(delay_ms) catch return error.Interrupted; - return true; -} - -test "isTransientServerError (#opencode-parity): overload/server_error retry; quota/invalid/auth do not" { - // transient server conditions → retry like a 5xx - try std.testing.expect(isTransientServerError("overloaded_error", null, "")); - try std.testing.expect(isTransientServerError("api_error", "server_error", "")); - try std.testing.expect(isTransientServerError("", "server_is_overloaded", "")); - try std.testing.expect(isTransientServerError("", null, "The server is Overloaded, please try again")); // case-insensitive, in message - // billing / input / auth → NOT transient, must hard-fail - try std.testing.expect(!isTransientServerError("insufficient_quota", "insufficient_quota", "You exceeded your current quota")); - try std.testing.expect(!isTransientServerError("invalid_request_error", null, "invalid prompt")); - try std.testing.expect(!isTransientServerError("authentication_error", null, "invalid api key")); -} - -/// #opencode-parity: a 429 body naming a billing/quota cap (OpenAI insufficient_quota, -/// "exceeded your current quota", "quota exceeded") — a usage limit a retry can't -/// clear, unlike transient rate-limit throttling — so we fail fast + fail over rather -/// than burning retry attempts. -fn isQuotaExceeded(body: []const u8) bool { - return std.ascii.indexOfIgnoreCase(body, "insufficient_quota") != null or - std.ascii.indexOfIgnoreCase(body, "insufficient quota") != null or - std.ascii.indexOfIgnoreCase(body, "exceeded your current quota") != null or - std.ascii.indexOfIgnoreCase(body, "quota exceeded") != null; -} - -test "isQuotaExceeded (#opencode-parity): billing cap detected, transient throttle not" { - try std.testing.expect(isQuotaExceeded("{\"error\":{\"code\":\"insufficient_quota\",\"message\":\"You exceeded your current quota\"}}")); - try std.testing.expect(isQuotaExceeded("Quota Exceeded for this key")); - // transient rate-limit -> NOT a quota cap; must still retry - try std.testing.expect(!isQuotaExceeded("Rate limit reached. Please try again in 20s.")); - try std.testing.expect(!isQuotaExceeded("429 too many requests")); -} - -test "isContextOverflow (#193/#203): matches structured code + every provider's phrasing, not unrelated errors" { - // codex/responses, openai, anthropic wire-format rejections all recover in-turn - try std.testing.expect(isContextOverflow("Your input exceeds the context window of 272000 tokens", null)); - try std.testing.expect(isContextOverflow("This model's maximum context length is 128000 tokens. However, you requested 130000", null)); - try std.testing.expect(isContextOverflow("context_length_exceeded", null)); - try std.testing.expect(isContextOverflow("prompt is too long: 219373 tokens > 200000 maximum", null)); - try std.testing.expect(isContextOverflow("input length and max_tokens exceed context limit", null)); - // #203: a structured error code recovers even when the message text is unfamiliar - // (a local / non-English provider whose phrasing we don't match on) - try std.testing.expect(isContextOverflow("de invoerlengte overschrijdt het venster", "context_length_exceeded")); - try std.testing.expect(isContextOverflow("", "context_window_exceeded")); - // unrelated API errors must NOT trigger a trim + retry, by message or by code - try std.testing.expect(!isContextOverflow("The API Key appears to be invalid or may have expired.", null)); - try std.testing.expect(!isContextOverflow("tool_choice is not supported", null)); - try std.testing.expect(!isContextOverflow("rate limit exceeded", null)); - try std.testing.expect(!isContextOverflow("model not found", null)); - try std.testing.expect(!isContextOverflow("some unrelated failure", "rate_limit_exceeded")); -} - -test "errorCode (#203): pulls root.error.code (openai/lmstudio), falls back to root.code, else null" { - var arena = std.heap.ArenaAllocator.init(std.testing.allocator); - defer arena.deinit(); - const a = arena.allocator(); - // openai/lmstudio/deepseek shape: {"error":{"code":"context_length_exceeded",...}} - var err: std.json.ObjectMap = .empty; - try err.put(a, "code", .{ .string = "context_length_exceeded" }); - var root1: std.json.ObjectMap = .empty; - try root1.put(a, "error", .{ .object = err }); - try std.testing.expectEqualStrings("context_length_exceeded", errorCode(root1).?); - // top-level code fallback - var root2: std.json.ObjectMap = .empty; - try root2.put(a, "code", .{ .string = "context_window_exceeded" }); - try std.testing.expectEqualStrings("context_window_exceeded", errorCode(root2).?); - // neither present → null (falls back to substring detection) - var root3: std.json.ObjectMap = .empty; - try root3.put(a, "message", .{ .string = "hi" }); - try std.testing.expect(errorCode(root3) == null); -} - -test "recordUsage (#202): floors the meter from the local estimate when usage is absent" { - var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator); - defer arena_state.deinit(); - const a = arena_state.allocator(); - - var msgs = std.json.Array.init(a); - var m: std.json.ObjectMap = .empty; - try m.put(a, "role", .{ .string = "user" }); - try m.put(a, "content", .{ .string = "the quick brown fox jumps over the lazy dog" }); - try msgs.append(.{ .object = m }); - - var agent: Agent = undefined; - agent.arena = a; - agent.messages = msgs; - agent.last_context_tokens = 0; - - // a response body with NO usage object previously froze the meter at its stale - // value; now it floors to max(full-input estimate, req_body_len/4) so the - // between-turns compaction gate can still fire. - const root: std.json.ObjectMap = .empty; - recordUsage(&agent, root, 4000); - - try std.testing.expect(agent.last_context_tokens > 0); - const expected = @max(fullInputEstimateTokens(&agent), @as(u64, 1000)); // 4000/4 - try std.testing.expectEqual(expected, agent.last_context_tokens); -} - -test "recoverContextOverflow (#193): overflow trims + retries once; guard and non-overflow fall through" { - var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator); - defer arena_state.deinit(); - const a = arena_state.allocator(); - // a runaway tool-loop history emergencyTrim can reclaim (mirrors the #163 shape: - // one clean user turn then only tool outputs, so trimOldestToolOutputs recovers) - var msgs = std.json.Array.init(a); - var um: std.json.ObjectMap = .empty; - try um.put(a, "role", .{ .string = "user" }); - try um.put(a, "content", .{ .string = "do a thing" }); - try msgs.append(.{ .object = um }); - const big = try a.alloc(u8, 5000); - @memset(big, 'x'); - var i: usize = 0; - while (i < 10) : (i += 1) { - var o: std.json.ObjectMap = .empty; - try o.put(a, "type", .{ .string = "function_call_output" }); - try o.put(a, "call_id", .{ .string = "c" }); - try o.put(a, "output", .{ .string = big }); - try msgs.append(.{ .object = o }); +const policy = @import("agent_request_policy.zig"); +const errorCode = policy.errorCode; +const isAuthError = policy.isAuthError; +const isQuotaExceeded = policy.isQuotaExceeded; +const recoverContextOverflow = policy.recoverContextOverflow; +const retryTransientServerError = policy.retryTransientServerError; + +const context = @import("agent_context.zig"); +pub const recordUsage = context.recordUsage; +pub const usageInt = context.usageInt; +pub const recordCost = context.recordCost; +pub const fullInputEstimateTokens = context.fullInputEstimateTokens; +pub const fullRequestEstimateTokens = context.fullRequestEstimateTokens; +pub const contextEstimate = context.contextEstimate; +pub const contextEstimateFromInputBytes = context.contextEstimateFromInputBytes; +pub const effectiveContextTokens = context.effectiveContextTokens; +pub const pairContextMeterWithCurrentLocal = context.pairContextMeterWithCurrentLocal; +pub const rebaseContextMeter = context.rebaseContextMeter; +pub const inputOverCompactThreshold = context.inputOverCompactThreshold; +pub const recordUsageResponses = context.recordUsageResponses; + +const responses = @import("agent_responses.zig"); +pub const ResponsesFailure = responses.ResponsesFailure; +pub const ResponsesResult = responses.ResponsesResult; +pub const parseResponses = responses.parseResponses; +pub const errorMessage = responses.errorMessage; + +pub const buildBody = @import("agent_request_body.zig").buildBody; + +/// Keep the normal request hot path allocation-free while avoiding a permanent +/// RSS high-water mark after one anomalously large stream. Small scratch arenas +/// retain their pages for the next request; large ones return all pages to the +/// backing allocator. History never lives here, so either reset mode is safe at +/// the start of the next request. +const scratch_retain_limit = 4 * 1024 * 1024; + +fn resetRequestScratch(scratch: *std.heap.ArenaAllocator) void { + if (scratch.queryCapacity() > scratch_retain_limit) { + _ = scratch.reset(.free_all); + } else { + _ = scratch.reset(.retain_capacity); } - var agent: Agent = undefined; - agent.arena = a; - agent.messages = msgs; - agent.tracer = null; - agent.provider = .{ .id = "anthropic", .kind = .anthropic, .auth = .x_api_key, .url = "", .api_key = "", .model = "claude", .context = 100000 }; - agent.last_context_tokens = 0; - - // overflow + trimmable history -> recovers (retry the turn), guard flips - var retried = false; - try std.testing.expect(recoverContextOverflow(&agent, "prompt is too long: 999 tokens > 100 maximum", null, &retried)); - try std.testing.expect(retried); - // a second overflow this request -> guard blocks a re-trim (no loop), but the - // meter stays pinned to the window so the between-turns compaction still engages - try std.testing.expect(!recoverContextOverflow(&agent, "prompt is too long", null, &retried)); - try std.testing.expectEqual(agent.provider.context, agent.last_context_tokens); - // an unrelated error never recovers, regardless of the guard - var retried2 = false; - try std.testing.expect(!recoverContextOverflow(&agent, "invalid api key", null, &retried2)); - try std.testing.expect(!retried2); -} - -test "fullInputEstimateTokens (#174): counts retained reasoning the chained usage never reports" { - var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator); - defer arena_state.deinit(); - const a = arena_state.allocator(); - var msgs = std.json.Array.init(a); - var agent: Agent = undefined; - agent.messages = msgs; - try std.testing.expectEqual(@as(u64, 0), fullInputEstimateTokens(&agent) / 100); // empty history ≈ nothing - // a fat encrypted-reasoning item — exactly what a WS-chained total_tokens excludes - const blob = "{\"type\":\"reasoning\",\"encrypted_content\":\"" ++ ("A" ** 8192) ++ "\"}"; - try msgs.append(try std.json.parseFromSliceLeaky(Value, a, blob, .{})); - agent.messages = msgs; - const est = fullInputEstimateTokens(&agent); - try std.testing.expect(est > 2000); // ~8KB serialized / 4 bytes-per-token } pub fn request(self: *Agent, tools: ?[]const u8) !std.json.ObjectMap { + // Startup paints the prompt while CA loading continues. The root turn and + // title task rendezvous here, then issue their requests concurrently. + http.waitForClientReady(self.io); + self.last_request_context_overflow = false; + self.last_request_write_failed = false; + self.last_usage_includes_output = false; var force = self.strict and tools != null; var stream_usage = true; // openai stream_options; dropped if rejected var auth_refreshed = false; // #148: at most one forced token refresh + retry @@ -332,7 +85,7 @@ pub fn request(self: *Agent, tools: ?[]const u8) !std.json.ObjectMap { // can use the scratch arena for this request. Safe: all scratch data is // consumed before the next request(); messages/todos/prompts live on the // session arena. - if (self.scratch_arena) |sa| _ = sa.reset(.retain_capacity); + if (self.scratch_arena) |sa| resetRequestScratch(sa); // #148: a login-sourced OAuth token (kimi/xai, ~1h) expires mid-session and // is minted only at startup; refresh it in place before the call when near // expiry so a long session — or a subagent that inherited the on-disk token — @@ -346,9 +99,10 @@ pub fn request(self: *Agent, tools: ?[]const u8) !std.json.ObjectMap { } } // #95: scrub any malformed function_call_output before it hits the wire. - sanitizeMessagesUtf8(self.arena, &self.messages); // invalid UTF-8 (any source/format) -> '?' so content never serializes as a byte-int array the API rejects - if (self.provider.kind == .responses) normalizeResponsesHistory(self.arena, &self.messages); - if (self.provider.kind == .openai) normalizeOpenAIHistory(self.arena, &self.messages); // #99: chat-completions sibling of the above + const message_arena = self.messageMutationAlloc(); + sanitizeMessagesUtf8(message_arena, &self.messages); // invalid UTF-8 (any source/format) -> '?' so content never serializes as a byte-int array the API rejects + if (self.provider.kind == .responses) normalizeResponsesHistory(message_arena, &self.messages); + if (self.provider.kind == .openai) normalizeOpenAIHistory(message_arena, &self.messages); // #99: chat-completions sibling of the above // #193 follow-up: bound any single oversized tool output (an uncapped MCP // result, a huge fetch on a small-window model) before send. The responses // path already hard-caps output above (normalizeResponsesHistory); this is the @@ -521,6 +275,7 @@ pub fn request(self: *Agent, tools: ?[]const u8) !std.json.ObjectMap { // record the real reason so the failed turn's --json error // event and trajectory node preserve it (#86). self.last_api_error = std.fmt.allocPrint(self.arena, "network error: {s} (gave up after {d} attempts)", .{ @errorName(err), max_attempts }) catch null; + self.last_request_write_failed = std.mem.eql(u8, @errorName(err), "WriteFailed"); if (telemetry.g_telem) |t| t.errorEvent("net", @errorName(err)); if (self.tracer) |tr| tr.api(self.label, self.provider.model, 0, body.len, 0, 0, 0, true); return error.ApiError; @@ -540,12 +295,14 @@ pub fn request(self: *Agent, tools: ?[]const u8) !std.json.ObjectMap { }; switch (r) { .ok => |obj| { + if (!self.compaction_request) self.compact_transport_failures = 0; self.recordUsageResponses(obj, body.len); if (self.tracer) |tr| tr.api(self.label, self.provider.model, ms, body.len, resp_body.len, self.last_context_tokens, self.last_cache_read, false); if (main_mod.json_mode and !self.sub) self.emit(.{ .type = "model_call_finished", .provider = self.provider.id, .model = self.provider.model, .ok = true, .ms = ms }); return obj; }, - .err => |msg| { + .err => |failure| { + const msg = failure.message; // (#codex-ws) Belt-and-braces mirror of openai/codex: server // rejected previous_response_id (stale WS session it no // longer recognizes) — close it and retry once with full @@ -564,23 +321,11 @@ pub fn request(self: *Agent, tools: ?[]const u8) !std.json.ObjectMap { // session would wedge (every retry resends the same // oversized history). Pin the meter to the window so the // ApiError compact-and-recover path engages. - if (isContextOverflow(msg, null)) { - self.last_context_tokens = self.provider.context; - // #193: the local pre-send gate uses a byte/4 LOWER bound, so - // the backend can still reject an input it let through. A good - // harness recovers the turn invisibly instead of failing it: - // reclaim room in place and retry the SAME request once - // (opencode compactAfterOverflow + rebuild). emergencyTrim does - // no summary request, so this can't reenter request(); the guard - // means a second overflow falls through to the error below, so we - // never loop. closeCodexWs re-anchors so the retry sends the - // trimmed full input, not a stale WS delta. - if (!context_retried and self.emergencyTrim() > 0) { - context_retried = true; - self.closeCodexWs(); - if (self.tracer) |tr| tr.note("context", "input over the window — emergency-trimmed and retrying the turn"); - continue :rebuild; - } + if (recoverContextOverflow(self, msg, failure.code, &context_retried)) { + // Responses may have sent a previous_response_id delta. + // Re-anchor after the shared recovery trims full history. + self.closeCodexWs(); + continue :rebuild; } if (self.tracer) |tr| tr.api(self.label, self.provider.model, ms, body.len, resp_body.len, 0, 0, true); try self.sayApiError("codex api error: {s}", .{msg}); @@ -607,13 +352,14 @@ pub fn request(self: *Agent, tools: ?[]const u8) !std.json.ObjectMap { return error.ApiError; }; self.recordUsage(root, body.len); + if (!self.compaction_request) self.compact_transport_failures = 0; if (self.tracer) |tr| tr.api(self.label, self.provider.model, ms, body.len, resp_body.len, self.last_context_tokens, self.last_cache_read, false); if (main_mod.json_mode and !self.sub) self.emit(.{ .type = "model_call_finished", .provider = self.provider.id, .model = self.provider.model, .ok = true, .ms = ms }); return root; } } - const resp = std.json.parseFromSliceLeaky(Value, self.arena, resp_body, .{ + const resp = std.json.parseFromSliceLeaky(Value, self.messageMutationAlloc(), resp_body, .{ .allocate = .alloc_always, }) catch { try self.sayApiError("unparseable response: {s}", .{resp_body[0..@min(resp_body.len, 400)]}); @@ -676,429 +422,25 @@ pub fn request(self: *Agent, tools: ?[]const u8) !std.json.ObjectMap { } self.recordUsage(root, body.len); + if (!self.compaction_request) self.compact_transport_failures = 0; if (self.tracer) |tr| tr.api(self.label, self.provider.model, ms, body.len, resp_body.len, self.last_context_tokens, self.last_cache_read, false); if (main_mod.json_mode and !self.sub) self.emit(.{ .type = "model_call_finished", .provider = self.provider.id, .model = self.provider.model, .ok = true, .ms = ms }); return root; } } -pub fn recordUsage(self: *Agent, root: std.json.ObjectMap, req_body_len: usize) void { - self.last_cache_read = 0; - // #202: keep the context meter live when the provider omits usage — otherwise - // the between-turns compaction gate freezes at a stale value and a long session - // can wedge. Mirror the codex/.responses fallback (req_body_len/4, floored at the - // full-input estimate) that recordUsageResponses already applies (#174). - const usage = root.get("usage") orelse return floorContextTokens(self, req_body_len / 4); - if (usage != .object) return floorContextTokens(self, req_body_len / 4); - const u = usage.object; - switch (self.provider.kind) { - .anthropic => { - var total: i64 = 0; - const fields = [_][]const u8{ - "input_tokens", "output_tokens", - "cache_read_input_tokens", "cache_creation_input_tokens", - }; - for (fields) |f| total += usageInt(u, f); - if (total > 0) self.last_context_tokens = @intCast(total); - const cache = usageInt(u, "cache_read_input_tokens"); - if (cache > 0) self.last_cache_read = @intCast(cache); - // cache writes bill ~like input; fold them into uncached input. - self.recordCost(usageInt(u, "input_tokens") + usageInt(u, "cache_creation_input_tokens"), cache, usageInt(u, "output_tokens")); - }, - .openai => { - if (usageInt(u, "total_tokens") > 0) self.last_context_tokens = @intCast(usageInt(u, "total_tokens")); - // deepseek reports prompt_cache_hit_tokens; the OpenAI shape - // nests cached_tokens under prompt_tokens_details. - var cache = usageInt(u, "prompt_cache_hit_tokens"); - if (cache == 0) if (u.get("prompt_tokens_details")) |d| if (d == .object) { - cache = usageInt(d.object, "cached_tokens"); - }; - if (cache > 0) self.last_cache_read = @intCast(cache); - self.recordCost(usageInt(u, "prompt_tokens") - cache, cache, usageInt(u, "completion_tokens")); - }, - // codex uses recordUsageResponses on its own path. - .responses => {}, - } -} - -/// #202: floor the context meter at the local estimate (full-input byte/4, or the -/// request-body byte/4 when the history isn't serialized yet) when the API omits -/// usage, so auto-compaction still triggers. Never lowers an existing higher count. -fn floorContextTokens(self: *Agent, est: u64) void { - const floor = @max(self.fullInputEstimateTokens(), est); - if (floor > self.last_context_tokens) self.last_context_tokens = floor; -} - -/// An integer usage field, or 0 if absent / wrong type. -pub fn usageInt(obj: std.json.ObjectMap, name: []const u8) i64 { - if (obj.get(name)) |v| if (v == .integer) return v.integer; - return 0; -} - -/// Record one request's usage into the session-wide tally (g_cost): -/// token counts always; USD only for API-key providers with a -/// price_table row. Subscription providers (codex, claude) bill flat -/// and tally as sub_calls; unpriced models as unpriced_calls. -pub fn recordCost(self: *Agent, uncached_in: i64, cache_in: i64, out: i64) void { - g_cost.add(self.io, self.provider.id, self.provider.model, uncached_in, cache_in, out); -} - -pub const ResponsesResult = union(enum) { ok: std.json.ObjectMap, err: []const u8 }; - -/// Pull the final `response` object out of a Codex SSE stream. Scans -/// `data:` lines for the last `response.completed`/`response.incomplete` -/// event; reports `response.failed`/`error` events or a plain JSON error -/// body as an error. -pub fn parseResponses(self: *Agent, body: []const u8) !ResponsesResult { - // The final output items arrive as individual `response.output_item.done` - // events; `response.completed` carries usage but an empty output array. - // Collect the done-items and synthesize a {output, usage} object. - // - // #124 slice 2b: every SSE event of the stream parses on the per-request - // scratch arena — the text deltas are the bulk of the body and their parse - // trees used to pile up on the session arena forever (~30-45 KB/turn - // measured). Only what escapes this request is detached onto the session - // arena: the output items (stepResponses appends them to history verbatim) - // via dupeJsonValue, and the usage/id/error scalars. - const scratch = self.scratchAlloc(); - var items = std.json.Array.init(self.arena); - var usage: ?Value = null; - var resp_id: ?[]const u8 = null; // response.id, for previous_response_id delta continuation (#codex-ws) - var saw_completed = false; - var err_msg: ?[]const u8 = null; - var it = std.mem.tokenizeScalar(u8, body, '\n'); - while (it.next()) |raw_line| { - const line = std.mem.trim(u8, raw_line, " \r"); - if (!std.mem.startsWith(u8, line, "data:")) continue; - const payload = std.mem.trim(u8, line["data:".len..], " "); - if (payload.len == 0 or std.mem.eql(u8, payload, "[DONE]")) continue; - const v = std.json.parseFromSliceLeaky(Value, scratch, payload, .{ .allocate = .alloc_always }) catch continue; - if (v != .object) continue; - const t = v.object.get("type") orelse continue; - if (t != .string) continue; - if (std.mem.eql(u8, t.string, "response.output_item.done")) { - if (v.object.get("item")) |item| try items.append(try util.dupeJsonValue(self.arena, item)); - } else if (std.mem.eql(u8, t.string, "response.completed") or std.mem.eql(u8, t.string, "response.incomplete")) { - saw_completed = true; - if (v.object.get("response")) |r| if (r == .object) { - if (r.object.get("usage")) |u| usage = try util.dupeJsonValue(self.arena, u); - if (r.object.get("id")) |idv| if (idv == .string) { - resp_id = try self.arena.dupe(u8, idv.string); - }; - }; - } else if (std.mem.eql(u8, t.string, "response.failed") or std.mem.eql(u8, t.string, "error")) { - err_msg = if (errorMessage(v.object)) |m| - self.arena.dupe(u8, m) catch "codex stream reported a failure" - else - "codex stream reported a failure"; - } - } - if (saw_completed or items.items.len > 0) { - var resp: std.json.ObjectMap = .empty; - try resp.put(self.arena, "output", .{ .array = items }); - if (usage) |u| try resp.put(self.arena, "usage", u); - if (resp_id) |rid| try resp.put(self.arena, "id", .{ .string = rid }); - return .{ .ok = resp }; - } - if (err_msg) |m| return .{ .err = m }; - // Not an SSE stream — maybe a JSON error body (401, rate limit, …). The - // message is duped off the scratch parse: sayApiError re-formats it onto - // the session arena, but the retry loop may also inspect it after another - // rebuild iteration reset the scratch arena. - const v = std.json.parseFromSliceLeaky(Value, scratch, body, .{ .allocate = .alloc_always }) catch return error.Unparseable; - if (v == .object) if (errorMessage(v.object)) |m| - return .{ .err = self.arena.dupe(u8, m) catch "unparseable provider error" }; - return error.Unparseable; -} - -pub fn errorMessage(obj: std.json.ObjectMap) ?[]const u8 { - if (obj.get("error")) |e| { - if (e == .object) { - if (e.object.get("message")) |m| if (m == .string) return m.string; - } else if (e == .string) return e.string; - } - if (obj.get("response")) |r| if (r == .object) { - if (r.object.get("error")) |e| if (e == .object) { - if (e.object.get("message")) |m| if (m == .string) return m.string; - }; - }; - if (obj.get("detail")) |d| if (d == .string) return d.string; - if (obj.get("message")) |m| if (m == .string) return m.string; - return null; -} - -/// #174: ~4-bytes/token estimate of the FULL history serialized as Responses -/// `input` items — the cost of the next full-history resend (runTurn closes -/// the WS per turn, so every turn's first request replays everything). The -/// chained WS usage can sit far below this: with previous_response_id the -/// server discards prior-turn reasoning from context, while a resend pays for -/// every retained encrypted reasoning item again. Counting discard writer — -/// no allocation. -pub fn fullInputEstimateTokens(self: *Agent) u64 { - var buf: [512]u8 = undefined; - var d: Io.Writer.Discarding = .init(&buf); - var s: std.json.Stringify = .{ .writer = &d.writer }; - s.write(Value{ .array = self.messages }) catch return 0; - return d.fullCount() / 4; -} - -/// Pre-send overflow gate (#193). `last_context_tokens` only updates from the -/// server's returned usage, so it lags tool output appended *within* a turn: a -/// burst of large tool results can push this turn's input past the model's wall -/// before the between-turns 80% meter ever sees it (the "input exceeds the -/// context window" rejection on codex/gpt-5.x). Estimating the full input -/// locally before each request lets the turn loop compact first — the analogue -/// of codex's run_pre_sampling_compact / opencode's isOverflow-before-each-turn. -/// Guarded on a known window (compactAt()==0 → don't compact blindly). -pub fn inputOverCompactThreshold(self: *Agent) bool { - const threshold = self.provider.compactAt(); - if (threshold == 0) return false; - // #203: fullInputEstimateTokens omits the ever-present system prompt + tool - // schemas, so it undercounts the real input and this gate under-fires. Add a - // baseline for that fixed prefill, clamped to 1/8 of the window so it can never - // dominate a small (local) window. (Kept out of fullInputEstimateTokens itself, - // which must stay pure over self.messages for the unit tests.) - const prefill_baseline_tokens: u64 = 8000; - const prefill = @min(prefill_baseline_tokens, self.provider.context / 8); - return fullInputEstimateTokens(self) + prefill >= threshold; -} - -test "inputOverCompactThreshold (#193): local estimate gates a pre-send compact" { - var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator); - defer arena_state.deinit(); - const a = arena_state.allocator(); - var msgs = std.json.Array.init(a); - var agent: Agent = undefined; - agent.messages = msgs; - // small window: compactAt() = 10_000/10*8 = 8_000 tokens ≈ 32_000 serialized bytes - agent.provider = .{ .id = "codex", .kind = .responses, .auth = .bearer, .url = "", .api_key = "", .model = "gpt-5", .context = 10_000 }; - try std.testing.expect(!inputOverCompactThreshold(&agent)); // empty history → under - // one fat tool output (~40KB serialized ≈ 10k est tokens) crosses 8k in a single append - const big = "{\"type\":\"function_call_output\",\"output\":\"" ++ ("x" ** 40000) ++ "\"}"; - try msgs.append(try std.json.parseFromSliceLeaky(Value, a, big, .{})); - agent.messages = msgs; - try std.testing.expect(inputOverCompactThreshold(&agent)); // burst → over → compact before send - // unknown window (context 0) never gates - agent.provider.context = 0; - try std.testing.expect(!inputOverCompactThreshold(&agent)); -} +test "request scratch retains normal capacity but releases an oversized spike" { + var scratch = std.heap.ArenaAllocator.init(std.testing.allocator); + defer scratch.deinit(); -pub fn recordUsageResponses(self: *Agent, response: std.json.ObjectMap, req_body_len: usize) void { - self.last_cache_read = 0; - // Fallback estimate (~4 bytes/token) from the serialized request body, - // which holds the full conversation — keeps the ctx meter and the - // compact@ trigger live when codex omits usage counts entirely. - const est: u64 = req_body_len / 4; - const usage = response.get("usage") orelse { - if (est > 0) self.last_context_tokens = est; - return; - }; - if (usage != .object) { - if (est > 0) self.last_context_tokens = est; - return; - } - const u = usage.object; - const in_tokens = usageInt(u, "input_tokens"); - const out_tokens = usageInt(u, "output_tokens"); - const total_tokens = usageInt(u, "total_tokens"); - if (total_tokens > 0) { - self.last_context_tokens = @intCast(total_tokens); - } else { - // Some Codex/Responses builds report only input/output counts. - // Still surface the prompt token counter instead of leaving the - // prompt stuck at "model · sub" with no context usage. - const computed_total = in_tokens + out_tokens; - if (computed_total > 0) { - self.last_context_tokens = @intCast(computed_total); - } else if (est > 0) { - self.last_context_tokens = est; - } - } - // #174: the server's chained number undercounts what the next full-history - // resend will cost, and the resend is the request that gets rejected — so - // the meter must never sit below the local full-input estimate. Same - // correction codex CLI applies (get_non_last_reasoning_items_tokens). - // Without it a long Extra-high session reads "95k/270k" right up until the - // backend rejects the resend for exceeding the window, and auto-compaction - // (gated on this meter) never rescues it. - const full_est = fullInputEstimateTokens(self); - if (full_est > self.last_context_tokens) self.last_context_tokens = full_est; - var cached: i64 = 0; - if (u.get("input_tokens_details")) |d| if (d == .object) { - cached = usageInt(d.object, "cached_tokens"); - if (cached > 0) self.last_cache_read = @intCast(cached); - }; - self.recordCost(in_tokens - cached, cached, out_tokens); -} + _ = try scratch.allocator().alloc(u8, 1024); + const normal_capacity = scratch.queryCapacity(); + try std.testing.expect(normal_capacity > 0); + resetRequestScratch(&scratch); + try std.testing.expectEqual(normal_capacity, scratch.queryCapacity()); -pub fn buildBody(self: *Agent, tools: ?[]const u8, force_tool: bool, stream: bool, stream_usage: bool) ![]u8 { - var aw: Io.Writer.Allocating = .init(self.gpa); - errdefer aw.deinit(); - var s: std.json.Stringify = .{ .writer = &aw.writer }; - try s.beginObject(); - try s.objectField("model"); - try s.write(self.provider.model); - switch (self.provider.kind) { - .anthropic => { - try s.objectField("max_tokens"); - try s.write(max_tokens); - if (stream) { - try s.objectField("stream"); - try s.write(true); - } - // Forced tool_choice conflicts with adaptive thinking; skip - // thinking only when forcing. - if (!force_tool) { - try s.objectField("thinking"); - try s.print("{s}", .{"{\"type\":\"adaptive\"}"}); - } - // Prompt caching (Anthropic): a cache_control breakpoint on the - // system block caches the whole stable prefix (system + tools). - // Must be block-level — a top-level cache_control is invalid. - // Other anthropic-format providers (minimax) get a plain - // string, since cache_control isn't part of their API. - try s.objectField("system"); - const cc = "{\"type\":\"ephemeral\"}"; - if (std.mem.eql(u8, self.provider.id, "anthropic")) { - try s.beginArray(); - try s.beginObject(); - try s.objectField("type"); - try s.write("text"); - try s.objectField("text"); - try s.write(self.systemPrompt()); - try s.objectField("cache_control"); - try s.print("{s}", .{cc}); - try s.endObject(); - try s.endArray(); - } else { - try s.write(self.systemPrompt()); - } - if (tools) |t| { - try s.objectField("tools"); - try s.print("{s}", .{t}); - if (force_tool) { - try s.objectField("tool_choice"); - try s.print("{s}", .{"{\"type\":\"any\"}"}); - } - } - try s.objectField("messages"); - // Cache the conversation prefix too (not just system) on the real - // Anthropic API: a rolling cache_control breakpoint on the last - // message. minimax (anthropic-format, no cache_control) is excluded. - const cache_msgs = std.mem.eql(u8, self.provider.id, "anthropic"); - try writeAnthropicMessages(&s, self.messages, cache_msgs); - }, - .openai => { - // graff's MakeOpenAiCompat: OpenAI deprecated max_tokens in - // favor of max_completion_tokens — send the new name to the - // direct OpenAI API, and to any provider that rejected the - // old one (cap_new, learned via the retry in request()). - const cap_field = if (std.mem.eql(u8, self.provider.id, "openai") or self.cap_new) - "max_completion_tokens" - else - "max_tokens"; - try s.objectField(cap_field); - try s.write(max_tokens); - if (stream) { - try s.objectField("stream"); - try s.write(true); - // Without include_usage the stream carries no token - // counts (context tracking + auto-compaction need them). - if (stream_usage) { - try s.objectField("stream_options"); - try s.print("{s}", .{"{\"include_usage\":true}"}); - } - } - if (tools) |t| { - try s.objectField("tools"); - try s.print("{s}", .{t}); - if (force_tool) { - try s.objectField("tool_choice"); - try s.write("required"); - } - } - try s.objectField("messages"); - try s.beginArray(); - try s.beginObject(); - try s.objectField("role"); - try s.write("system"); - try s.objectField("content"); - try s.write(self.systemPrompt()); - try s.endObject(); - for (self.messages.items) |m| try writeOpenAIMessageNormalized(&s, m); - try s.endArray(); - // Reasoning-effort hint for OpenAI-compatible providers that - // honor it (codegraff gateway, deepseek). Mirrors the - // Responses `reasoning.effort` set in the branch below. - if (self.effortApplies() and !self.effort_rejected) { - try s.objectField("reasoning_effort"); - try s.write(if (self.reasoning == .ultra) "max" else @tagName(self.reasoning)); - } - // Kimi K2.7's model card recommends temperature 1.0 + top_p 0.95 - // for its (always-on) Thinking mode; graff otherwise leaves - // sampling to the server default. - if (std.mem.eql(u8, self.provider.id, "kimi")) { - try s.objectField("temperature"); - try s.write(@as(f64, 1.0)); - try s.objectField("top_p"); - try s.write(@as(f64, 0.95)); - } - }, - .responses => { - // Codex / ChatGPT Responses API. system prompt → instructions; - // history items are valid input items; stream is required by - // the backend (we buffer + parse the SSE). reasoning items are - // returned encrypted and passed back for cross-turn continuity. - try s.objectField("instructions"); - try s.write(self.systemPrompt()); - // Codex WS delta: once a response.id is held on a live WS session, - // send previous_response_id + only the items the server does not yet - // hold, instead of the full history (avoids the huge frame that the - // backend rejects → WriteFailed). Full input otherwise (first turn/SSE). - if (self.codex_ws != null) if (self.codex_prev_id) |pid| { - try s.objectField("previous_response_id"); - try s.write(pid); - }; - try s.objectField("input"); - if (self.codex_ws != null and self.codex_prev_id != null and self.codex_sent_upto <= self.messages.items.len) { - var delta = std.json.Array.init(self.arena); - for (self.messages.items[self.codex_sent_upto..]) |m| try delta.append(m); - try s.write(Value{ .array = delta }); - } else { - try s.write(Value{ .array = self.messages }); - } - if (tools) |t| { - try s.objectField("tools"); - try s.print("{s}", .{t}); - try s.objectField("tool_choice"); - try s.write(if (force_tool) "required" else "auto"); - try s.objectField("parallel_tool_calls"); - try s.write(true); - } - // Codex "fast" mode (/fast): request the priority service - // tier for lower latency. This branch is codex-only, so it is - // never emitted for other providers. - if (self.fast) { - try s.objectField("service_tier"); - try s.write("priority"); - } - try s.objectField("reasoning"); - try s.beginObject(); - try s.objectField("effort"); - // Ultra is a Graff/Codex client preset: maximum server reasoning - // plus automatic delegation. The backend wire value is `max`. - try s.write(if (self.reasoning == .ultra) "max" else @tagName(self.reasoning)); - try s.endObject(); - try s.objectField("include"); - try s.beginArray(); - try s.write("reasoning.encrypted_content"); - try s.endArray(); - try s.objectField("store"); - try s.write(false); - try s.objectField("stream"); - try s.write(true); - }, - } - try s.endObject(); - return aw.toOwnedSlice(); + _ = try scratch.allocator().alloc(u8, scratch_retain_limit + 1); + try std.testing.expect(scratch.queryCapacity() > scratch_retain_limit); + resetRequestScratch(&scratch); + try std.testing.expectEqual(@as(usize, 0), scratch.queryCapacity()); } diff --git a/src/agent_request_body.zig b/src/agent_request_body.zig new file mode 100644 index 00000000..6d0266fc --- /dev/null +++ b/src/agent_request_body.zig @@ -0,0 +1,184 @@ +//! Provider-specific request-body serialization. + +const std = @import("std"); +const Io = std.Io; +const Value = std.json.Value; + +const main_mod = @import("main.zig"); +const max_tokens = main_mod.max_tokens; +const Agent = @import("agent.zig").Agent; +const serde = @import("serde.zig"); +const writeAnthropicMessages = serde.writeAnthropicMessages; +const writeOpenAIMessageNormalized = serde.writeOpenAIMessageNormalized; + +pub fn buildBody(self: *Agent, tools: ?[]const u8, force_tool: bool, stream: bool, stream_usage: bool) ![]u8 { + var aw: Io.Writer.Allocating = .init(self.gpa); + errdefer aw.deinit(); + var s: std.json.Stringify = .{ .writer = &aw.writer }; + try s.beginObject(); + try s.objectField("model"); + try s.write(self.provider.model); + switch (self.provider.kind) { + .anthropic => { + try s.objectField("max_tokens"); + try s.write(max_tokens); + if (stream) { + try s.objectField("stream"); + try s.write(true); + } + // Forced tool_choice conflicts with adaptive thinking; skip + // thinking only when forcing. + if (!force_tool) { + try s.objectField("thinking"); + try s.print("{s}", .{"{\"type\":\"adaptive\"}"}); + } + // Prompt caching (Anthropic): a cache_control breakpoint on the + // system block caches the whole stable prefix (system + tools). + // Must be block-level — a top-level cache_control is invalid. + // Other anthropic-format providers (minimax) get a plain + // string, since cache_control isn't part of their API. + try s.objectField("system"); + const cc = "{\"type\":\"ephemeral\"}"; + if (std.mem.eql(u8, self.provider.id, "anthropic")) { + try s.beginArray(); + try s.beginObject(); + try s.objectField("type"); + try s.write("text"); + try s.objectField("text"); + try s.write(self.systemPrompt()); + try s.objectField("cache_control"); + try s.print("{s}", .{cc}); + try s.endObject(); + try s.endArray(); + } else { + try s.write(self.systemPrompt()); + } + if (tools) |t| { + try s.objectField("tools"); + try s.print("{s}", .{t}); + if (force_tool) { + try s.objectField("tool_choice"); + try s.print("{s}", .{"{\"type\":\"any\"}"}); + } + } + try s.objectField("messages"); + // Cache the conversation prefix too (not just system) on the real + // Anthropic API: a rolling cache_control breakpoint on the last + // message. minimax (anthropic-format, no cache_control) is excluded. + const cache_msgs = std.mem.eql(u8, self.provider.id, "anthropic"); + try writeAnthropicMessages(&s, self.messages, cache_msgs); + }, + .openai => { + // graff's MakeOpenAiCompat: OpenAI deprecated max_tokens in + // favor of max_completion_tokens — send the new name to the + // direct OpenAI API, and to any provider that rejected the + // old one (cap_new, learned via the retry in request()). + const cap_field = if (std.mem.eql(u8, self.provider.id, "openai") or self.cap_new) + "max_completion_tokens" + else + "max_tokens"; + try s.objectField(cap_field); + try s.write(max_tokens); + if (stream) { + try s.objectField("stream"); + try s.write(true); + // Without include_usage the stream carries no token + // counts (context tracking + auto-compaction need them). + if (stream_usage) { + try s.objectField("stream_options"); + try s.print("{s}", .{"{\"include_usage\":true}"}); + } + } + if (tools) |t| { + try s.objectField("tools"); + try s.print("{s}", .{t}); + if (force_tool) { + try s.objectField("tool_choice"); + try s.write("required"); + } + } + try s.objectField("messages"); + try s.beginArray(); + try s.beginObject(); + try s.objectField("role"); + try s.write("system"); + try s.objectField("content"); + try s.write(self.systemPrompt()); + try s.endObject(); + for (self.messages.items) |m| try writeOpenAIMessageNormalized(&s, m); + try s.endArray(); + // Reasoning-effort hint for OpenAI-compatible providers that + // honor it (codegraff gateway, deepseek). Mirrors the + // Responses `reasoning.effort` set in the branch below. + if (self.effortApplies() and !self.effort_rejected) { + try s.objectField("reasoning_effort"); + try s.write(if (self.reasoning == .ultra) "max" else @tagName(self.reasoning)); + } + // Kimi K2.7's model card recommends temperature 1.0 + top_p 0.95 + // for its (always-on) Thinking mode; graff otherwise leaves + // sampling to the server default. + if (std.mem.eql(u8, self.provider.id, "kimi")) { + try s.objectField("temperature"); + try s.write(@as(f64, 1.0)); + try s.objectField("top_p"); + try s.write(@as(f64, 0.95)); + } + }, + .responses => { + // Codex / ChatGPT Responses API. system prompt → instructions; + // history items are valid input items; stream is required by + // the backend (we buffer + parse the SSE). reasoning items are + // returned encrypted and passed back for cross-turn continuity. + try s.objectField("instructions"); + try s.write(self.systemPrompt()); + // Codex WS delta: once a response.id is held on a live WS session, + // send previous_response_id + only the items the server does not yet + // hold, instead of the full history (avoids the huge frame that the + // backend rejects → WriteFailed). Full input otherwise (first turn/SSE). + if (self.codex_ws != null) if (self.codex_prev_id) |pid| { + try s.objectField("previous_response_id"); + try s.write(pid); + }; + try s.objectField("input"); + if (self.codex_ws != null and self.codex_prev_id != null and self.codex_sent_upto <= self.messages.items.len) { + var delta = std.json.Array.init(self.arena); + for (self.messages.items[self.codex_sent_upto..]) |m| try delta.append(m); + try s.write(Value{ .array = delta }); + } else { + try s.write(Value{ .array = self.messages }); + } + if (tools) |t| { + try s.objectField("tools"); + try s.print("{s}", .{t}); + try s.objectField("tool_choice"); + try s.write(if (force_tool) "required" else "auto"); + try s.objectField("parallel_tool_calls"); + try s.write(true); + } + // Codex "fast" mode (/fast): request the priority service + // tier for lower latency. This branch is codex-only, so it is + // never emitted for other providers. + if (self.fast) { + try s.objectField("service_tier"); + try s.write("priority"); + } + try s.objectField("reasoning"); + try s.beginObject(); + try s.objectField("effort"); + // Ultra is a Graff/Codex client preset: maximum server reasoning + // plus automatic delegation. The backend wire value is `max`. + try s.write(if (self.reasoning == .ultra) "max" else @tagName(self.reasoning)); + try s.endObject(); + try s.objectField("include"); + try s.beginArray(); + try s.write("reasoning.encrypted_content"); + try s.endArray(); + try s.objectField("store"); + try s.write(false); + try s.objectField("stream"); + try s.write(true); + }, + } + try s.endObject(); + return aw.toOwnedSlice(); +} diff --git a/src/agent_request_policy.zig b/src/agent_request_policy.zig new file mode 100644 index 00000000..19737dcb --- /dev/null +++ b/src/agent_request_policy.zig @@ -0,0 +1,294 @@ +//! Request retry and provider-error policy shared by the request loop. + +const std = @import("std"); + +const Agent = @import("agent.zig").Agent; +const http = @import("http.zig"); +const RetryPlan = http.RetryPlan; +const telemetry = @import("telemetry.zig"); + +pub fn isAuthError(msg: []const u8) bool { + return std.ascii.indexOfIgnoreCase(msg, "api key") != null or + std.ascii.indexOfIgnoreCase(msg, "unauthorized") != null or + std.ascii.indexOfIgnoreCase(msg, "expired") != null or + std.ascii.indexOfIgnoreCase(msg, "authentication") != null or + std.ascii.indexOfIgnoreCase(msg, "invalid_api_key") != null; +} + +test "isAuthError (#148): auth failures only, not credits/rate/other" { + // real provider 401 phrasings → refresh + retry + try std.testing.expect(isAuthError("The API Key appears to be invalid or may have expired.")); + try std.testing.expect(isAuthError("Unauthorized")); + try std.testing.expect(isAuthError("authentication_error")); + try std.testing.expect(isAuthError("invalid_api_key")); + // NOT auth — must never trigger a refresh loop + try std.testing.expect(!isAuthError("You have run out of credits or need a Grok subscription.")); + try std.testing.expect(!isAuthError("rate limit exceeded")); + try std.testing.expect(!isAuthError("model not found")); + try std.testing.expect(!isAuthError("context length exceeded")); +} + +/// True if `msg` is a provider's "input is over the context window" rejection, +/// across wire formats: codex/responses ("exceeds the context window"), openai +/// ("maximum context length", "context_length_exceeded"), anthropic ("prompt is +/// too long", "exceed context limit"). Drives the in-turn emergency-trim + retry +/// recovery symmetrically for every provider (#193) — before it, only the codex +/// path recovered and anthropic/openai died on an over-window turn. +fn isContextOverflow(msg: []const u8, code: ?[]const u8) bool { + // #203: match the structured error code first (openai/codex parity — a local or + // non-English provider whose message text differs still recovers), then fall back + // to the human-readable phrasing. + if (code) |c| { + const codes = [_][]const u8{ "context_length_exceeded", "context_window_exceeded" }; + for (codes) |k| if (std.mem.eql(u8, c, k)) return true; + } + const needles = [_][]const u8{ + "context window", // codex/responses: "exceeds the context window" + "context length", // openai: "maximum context length is N tokens" + "context_length_exceeded", // openai error code echoed into the message + "context limit", // anthropic: "input length and max_tokens exceed context limit" + "prompt is too long", // anthropic: "prompt is too long: N tokens > M maximum" + "maximum context", // defensive: "maximum context ... exceeded" + }; + for (needles) |n| if (std.mem.indexOf(u8, msg, n) != null) return true; + return false; +} + +/// The structured error code from a parsed error envelope, if any: openai / lmstudio / +/// deepseek put it at root.error.code; some providers use a top-level root.code (#203). +pub fn errorCode(root: std.json.ObjectMap) ?[]const u8 { + if (root.get("error")) |ev| { + if (ev == .object) { + if (ev.object.get("code")) |cv| { + if (cv == .string) return cv.string; + } + } + } + if (root.get("code")) |cv| { + if (cv == .string) return cv.string; + } + // Responses failure events wrap the same error object under `response`. + if (root.get("response")) |rv| { + if (rv == .object) { + if (rv.object.get("error")) |ev| { + if (ev == .object) { + if (ev.object.get("code")) |cv| { + if (cv == .string) return cv.string; + } + } + } + if (rv.object.get("code")) |cv| { + if (cv == .string) return cv.string; + } + } + } + return null; +} + +/// #193 follow-up: shared in-turn context-overflow recovery for the three +/// anthropic/openai error branches (streamed error event, non-streamed +/// `{"type":"error"}` envelope, and the generic apiErrorMessage path). Before +/// this, only the codex/.responses branch recovered — anthropic/openai died on an +/// over-window turn. Returns true if the caller should `continue` the rebuild loop +/// (emergency-trimmed, retry the same request once); false to fall through to the +/// normal error. Pins the meter to the window FIRST so the between-turns +/// compaction engages even when we can't recover here (the rejected request +/// returns no usage to correct the lagging meter). Guarded by `retried` (one +/// `context_retried` shared across every branch of a request) so a second overflow +/// falls through and never loops. These wire formats send the full input each +/// rebuild, so — unlike the codex branch — no closeCodexWs re-anchor is needed. +pub fn recoverContextOverflow(self: *Agent, msg: []const u8, code: ?[]const u8, retried: *bool) bool { + if (!isContextOverflow(msg, code)) return false; + self.last_request_context_overflow = true; + // Rejection proves at least the advertised window, not an exact total. + // Preserve stronger server/local evidence already above that floor. + const estimate = self.contextEstimate(); + self.last_context_tokens = @max(estimate.effective, self.provider.context); + self.context_local_tokens = estimate.local; + // compact() marks its synthetic summary request explicitly. Generic + // in-request recovery is destructive there: the synthetic compact + // instruction is the newest clean user turn, so emergencyTrim can discard + // the entire real conversation and retry with only "summarize it". Let the + // outer compactOrRecover policy handle a failed summary after compact()'s + // errdefer removes that synthetic instruction. + if (self.compaction_request) return false; + if (retried.* or self.emergencyTrim() == 0) return false; + retried.* = true; + if (self.tracer) |tr| tr.note("context", "input over the window — emergency-trimmed and retrying the turn"); + return true; +} + +const max_server_retries: usize = 3; // #opencode-parity: bounded retries for a transient in-stream server overload + +/// #opencode-parity: an in-band error event (an SSE {"type":"error"} or a JSON +/// error envelope) naming a TRANSIENT server condition — Anthropic overloaded_error, +/// OpenAI server_error / server_is_overloaded, or plain "overloaded" — is a 5xx that +/// surfaced mid-stream and should be retried, not hard-failed. Billing / quota / +/// invalid-input errors are NOT transient and fall through to a hard fail. +fn isTransientServerError(etype: []const u8, code: ?[]const u8, msg: []const u8) bool { + const needles = [_][]const u8{ "overloaded", "server_error", "server_is_overloaded" }; + for (needles) |n| { + if (std.ascii.indexOfIgnoreCase(etype, n) != null) return true; + if (std.ascii.indexOfIgnoreCase(msg, n) != null) return true; + if (code) |c| if (std.ascii.indexOfIgnoreCase(c, n) != null) return true; + } + return false; +} + +/// If an in-stream error names a transient server overload, back off and retry the +/// request (bounded), like a 5xx — returns true to signal the caller to `continue`. +/// Esc during the backoff propagates as error.Interrupted. #opencode-parity. +pub fn retryTransientServerError(self: *Agent, etype: []const u8, code: ?[]const u8, msg: []const u8, retries: *usize) !bool { + if (!isTransientServerError(etype, code, msg)) return false; + if (retries.* >= max_server_retries) return false; + retries.* += 1; + self.partial_text.clearRetainingCapacity(); // fresh re-stream after the retry, no concat + const delay_ms = RetryPlan.delayMs(true, retries.* - 1); // 1·2·4s + try self.say("[server overloaded — retrying in {d}s ({d}/{d})]\n", .{ delay_ms / 1000, retries.*, max_server_retries }); + if (self.tracer) |tr| tr.note("retry", "server overloaded (in-stream)"); + if (telemetry.g_telem) |t| t.errorEvent("server_overloaded", if (msg.len > 0) msg else etype); + self.sleepInterruptible(delay_ms) catch return error.Interrupted; + return true; +} + +test "isTransientServerError (#opencode-parity): overload/server_error retry; quota/invalid/auth do not" { + // transient server conditions → retry like a 5xx + try std.testing.expect(isTransientServerError("overloaded_error", null, "")); + try std.testing.expect(isTransientServerError("api_error", "server_error", "")); + try std.testing.expect(isTransientServerError("", "server_is_overloaded", "")); + try std.testing.expect(isTransientServerError("", null, "The server is Overloaded, please try again")); // case-insensitive, in message + // billing / input / auth → NOT transient, must hard-fail + try std.testing.expect(!isTransientServerError("insufficient_quota", "insufficient_quota", "You exceeded your current quota")); + try std.testing.expect(!isTransientServerError("invalid_request_error", null, "invalid prompt")); + try std.testing.expect(!isTransientServerError("authentication_error", null, "invalid api key")); +} + +/// #opencode-parity: a 429 body naming a billing/quota cap (OpenAI insufficient_quota, +/// "exceeded your current quota", "quota exceeded") — a usage limit a retry can't +/// clear, unlike transient rate-limit throttling — so we fail fast + fail over rather +/// than burning retry attempts. +pub fn isQuotaExceeded(body: []const u8) bool { + return std.ascii.indexOfIgnoreCase(body, "insufficient_quota") != null or + std.ascii.indexOfIgnoreCase(body, "insufficient quota") != null or + std.ascii.indexOfIgnoreCase(body, "exceeded your current quota") != null or + std.ascii.indexOfIgnoreCase(body, "quota exceeded") != null; +} + +test "isQuotaExceeded (#opencode-parity): billing cap detected, transient throttle not" { + try std.testing.expect(isQuotaExceeded("{\"error\":{\"code\":\"insufficient_quota\",\"message\":\"You exceeded your current quota\"}}")); + try std.testing.expect(isQuotaExceeded("Quota Exceeded for this key")); + // transient rate-limit -> NOT a quota cap; must still retry + try std.testing.expect(!isQuotaExceeded("Rate limit reached. Please try again in 20s.")); + try std.testing.expect(!isQuotaExceeded("429 too many requests")); +} + +test "isContextOverflow (#193/#203): matches structured code + every provider's phrasing, not unrelated errors" { + // codex/responses, openai, anthropic wire-format rejections all recover in-turn + try std.testing.expect(isContextOverflow("Your input exceeds the context window of 272000 tokens", null)); + try std.testing.expect(isContextOverflow("This model's maximum context length is 128000 tokens. However, you requested 130000", null)); + try std.testing.expect(isContextOverflow("context_length_exceeded", null)); + try std.testing.expect(isContextOverflow("prompt is too long: 219373 tokens > 200000 maximum", null)); + try std.testing.expect(isContextOverflow("input length and max_tokens exceed context limit", null)); + // #203: a structured error code recovers even when the message text is unfamiliar + // (a local / non-English provider whose phrasing we don't match on) + try std.testing.expect(isContextOverflow("de invoerlengte overschrijdt het venster", "context_length_exceeded")); + try std.testing.expect(isContextOverflow("", "context_window_exceeded")); + // unrelated API errors must NOT trigger a trim + retry, by message or by code + try std.testing.expect(!isContextOverflow("The API Key appears to be invalid or may have expired.", null)); + try std.testing.expect(!isContextOverflow("tool_choice is not supported", null)); + try std.testing.expect(!isContextOverflow("rate limit exceeded", null)); + try std.testing.expect(!isContextOverflow("model not found", null)); + try std.testing.expect(!isContextOverflow("some unrelated failure", "rate_limit_exceeded")); +} + +test "errorCode (#203): pulls root.error.code (openai/lmstudio), falls back to root.code, else null" { + var arena = std.heap.ArenaAllocator.init(std.testing.allocator); + defer arena.deinit(); + const a = arena.allocator(); + // openai/lmstudio/deepseek shape: {"error":{"code":"context_length_exceeded",...}} + var err: std.json.ObjectMap = .empty; + try err.put(a, "code", .{ .string = "context_length_exceeded" }); + var root1: std.json.ObjectMap = .empty; + try root1.put(a, "error", .{ .object = err }); + try std.testing.expectEqualStrings("context_length_exceeded", errorCode(root1).?); + // top-level code fallback + var root2: std.json.ObjectMap = .empty; + try root2.put(a, "code", .{ .string = "context_window_exceeded" }); + try std.testing.expectEqualStrings("context_window_exceeded", errorCode(root2).?); + // neither present → null (falls back to substring detection) + var root3: std.json.ObjectMap = .empty; + try root3.put(a, "message", .{ .string = "hi" }); + try std.testing.expect(errorCode(root3) == null); + + // Responses stream shape: {response:{error:{code}}}. + var responses_error: std.json.ObjectMap = .empty; + try responses_error.put(a, "code", .{ .string = "context_length_exceeded" }); + var responses_payload: std.json.ObjectMap = .empty; + try responses_payload.put(a, "error", .{ .object = responses_error }); + var root4: std.json.ObjectMap = .empty; + try root4.put(a, "response", .{ .object = responses_payload }); + try std.testing.expectEqualStrings("context_length_exceeded", errorCode(root4).?); +} + +test "recoverContextOverflow (#193): overflow trims + retries once; guard and non-overflow fall through" { + var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator); + defer arena_state.deinit(); + const a = arena_state.allocator(); + // a runaway tool-loop history emergencyTrim can reclaim (mirrors the #163 shape: + // one clean user turn then only tool outputs, so trimOldestToolOutputs recovers) + var msgs = std.json.Array.init(a); + var um: std.json.ObjectMap = .empty; + try um.put(a, "role", .{ .string = "user" }); + try um.put(a, "content", .{ .string = "do a thing" }); + try msgs.append(.{ .object = um }); + const big = try a.alloc(u8, 5000); + @memset(big, 'x'); + var i: usize = 0; + while (i < 10) : (i += 1) { + var o: std.json.ObjectMap = .empty; + try o.put(a, "type", .{ .string = "function_call_output" }); + try o.put(a, "call_id", .{ .string = "c" }); + try o.put(a, "output", .{ .string = big }); + try msgs.append(.{ .object = o }); + } + var agent: Agent = undefined; + agent.arena = a; + agent.messages = msgs; + agent.tracer = null; + agent.provider = .{ .id = "anthropic", .kind = .anthropic, .auth = .x_api_key, .url = "", .api_key = "", .model = "claude", .context = 100000 }; + agent.last_context_tokens = 0; + agent.sub = false; + agent.strict = false; + agent.sys_normal = ""; + agent.sys_strict = ""; + agent.tools_anthropic = ""; + agent.context_local_tokens = agent.fullRequestEstimateTokens(); + agent.stream_quiet = true; + agent.compaction_request = true; + agent.last_request_context_overflow = false; + + // A compaction-summary request must never run generic emergency recovery: + // its synthetic user instruction would become a destructive trim boundary. + const before_quiet = agent.messages.items.len; + var quiet_retried = false; + try std.testing.expect(!recoverContextOverflow(&agent, "prompt is too long", null, &quiet_retried)); + try std.testing.expect(!quiet_retried); + try std.testing.expect(agent.last_request_context_overflow); + try std.testing.expectEqual(before_quiet, agent.messages.items.len); + try std.testing.expectEqual(agent.provider.context, agent.last_context_tokens); + agent.compaction_request = false; + agent.last_context_tokens = 0; + + // Quiet one-shot output is not compaction. It must still recover normally. + var retried = false; + try std.testing.expect(recoverContextOverflow(&agent, "prompt is too long: 999 tokens > 100 maximum", null, &retried)); + try std.testing.expect(retried); + // a second overflow this request -> guard blocks a re-trim (no loop), but the + // meter stays pinned to the window so the between-turns compaction still engages + try std.testing.expect(!recoverContextOverflow(&agent, "prompt is too long", null, &retried)); + try std.testing.expectEqual(agent.provider.context, agent.last_context_tokens); + // an unrelated error never recovers, regardless of the guard + var retried2 = false; + try std.testing.expect(!recoverContextOverflow(&agent, "invalid api key", null, &retried2)); + try std.testing.expect(!retried2); +} diff --git a/src/agent_responses.zig b/src/agent_responses.zig new file mode 100644 index 00000000..7ee0e0b8 --- /dev/null +++ b/src/agent_responses.zig @@ -0,0 +1,162 @@ +//! Reassembly of OpenAI Responses SSE events into a response object. + +const std = @import("std"); +const Value = std.json.Value; + +const Agent = @import("agent.zig").Agent; +const policy = @import("agent_request_policy.zig"); +const errorCode = policy.errorCode; +const util = @import("util.zig"); + +pub const ResponsesFailure = struct { + message: []const u8, + code: ?[]const u8 = null, +}; + +pub const ResponsesResult = union(enum) { ok: std.json.ObjectMap, err: ResponsesFailure }; + +pub fn parseResponses(self: *Agent, body: []const u8) !ResponsesResult { + // The final output items arrive as individual `response.output_item.done` + // events; `response.completed` carries usage but an empty output array. + // Collect the done-items and synthesize a {output, usage} object. + // + // #124 slice 2b: every SSE event of the stream parses on the per-request + // scratch arena — the text deltas are the bulk of the body and their parse + // trees used to pile up on the session arena forever (~30-45 KB/turn + // measured). Only what escapes this request is detached onto the session + // arena: the output items (stepResponses appends them to history verbatim) + // via dupeJsonValue, and the usage/id/error scalars. + const scratch = self.scratchAlloc(); + const result_arena = self.messageMutationAlloc(); + var items = std.json.Array.init(result_arena); + var usage: ?Value = null; + var resp_id: ?[]const u8 = null; // response.id, for previous_response_id delta continuation (#codex-ws) + var saw_completed = false; + var saw_incomplete = false; + var err_msg: ?[]const u8 = null; + var err_code: ?[]const u8 = null; + var it = std.mem.tokenizeScalar(u8, body, '\n'); + while (it.next()) |raw_line| { + const line = std.mem.trim(u8, raw_line, " \r"); + if (!std.mem.startsWith(u8, line, "data:")) continue; + const payload = std.mem.trim(u8, line["data:".len..], " "); + if (payload.len == 0 or std.mem.eql(u8, payload, "[DONE]")) continue; + const v = std.json.parseFromSliceLeaky(Value, scratch, payload, .{ .allocate = .alloc_always }) catch continue; + if (v != .object) continue; + const t = v.object.get("type") orelse continue; + if (t != .string) continue; + if (std.mem.eql(u8, t.string, "response.output_item.done")) { + if (v.object.get("item")) |item| try items.append(try util.dupeJsonValue(result_arena, item)); + } else if (std.mem.eql(u8, t.string, "response.completed") or std.mem.eql(u8, t.string, "response.incomplete")) { + saw_completed = std.mem.eql(u8, t.string, "response.completed"); + saw_incomplete = std.mem.eql(u8, t.string, "response.incomplete"); + if (v.object.get("response")) |r| if (r == .object) { + if (r.object.get("usage")) |u| usage = try util.dupeJsonValue(result_arena, u); + if (r.object.get("id")) |idv| if (idv == .string) { + resp_id = try result_arena.dupe(u8, idv.string); + }; + }; + } else if (std.mem.eql(u8, t.string, "response.failed") or std.mem.eql(u8, t.string, "error")) { + err_msg = if (errorMessage(v.object)) |m| + result_arena.dupe(u8, m) catch "codex stream reported a failure" + else + "codex stream reported a failure"; + if (errorCode(v.object)) |code| err_code = result_arena.dupe(u8, code) catch null; + } + } + // A terminal failure wins even if the stream produced partial completed + // items first. Committing those items as a successful compaction summary + // would replace live history with a response the provider explicitly + // rejected. + if (err_msg) |m| return .{ .err = .{ .message = m, .code = err_code } }; + if (saw_completed or saw_incomplete or items.items.len > 0) { + var resp: std.json.ObjectMap = .empty; + try resp.put(result_arena, "output", .{ .array = items }); + if (usage) |u| try resp.put(result_arena, "usage", u); + if (resp_id) |rid| try resp.put(result_arena, "id", .{ .string = rid }); + // Item-only streams are tolerated for ordinary turns, but are not a + // safe sole source for a compaction handoff: no terminal completed event + // means the transport may have ended mid-summary. + if (!saw_completed) try resp.put(result_arena, "incomplete", .{ .bool = true }); + return .{ .ok = resp }; + } + // Not an SSE stream — maybe a JSON error body (401, rate limit, …). The + // message is duped off the scratch parse: sayApiError re-formats it onto + // the session arena, but the retry loop may also inspect it after another + // rebuild iteration reset the scratch arena. + const v = std.json.parseFromSliceLeaky(Value, scratch, body, .{ .allocate = .alloc_always }) catch return error.Unparseable; + if (v == .object) { + const message = errorMessage(v.object); + const raw_code = errorCode(v.object); + if (message != null or raw_code != null) { + const code = if (raw_code) |c| result_arena.dupe(u8, c) catch null else null; + const m = message orelse "codex provider reported an error"; + return .{ .err = .{ + .message = result_arena.dupe(u8, m) catch "unparseable provider error", + .code = code, + } }; + } + } + return error.Unparseable; +} + +test "parseResponses: terminal failure beats partial items; incomplete stays marked" { + var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator); + defer arena_state.deinit(); + const a = arena_state.allocator(); + var agent: Agent = undefined; + agent.arena = a; + agent.scratch_arena = null; + agent.message_mutation_arena = null; + + const item_event = + "data: {\"type\":\"response.output_item.done\",\"item\":{\"type\":\"message\",\"role\":\"assistant\",\"content\":[{\"type\":\"output_text\",\"text\":\"partial\"}]}}\n"; + const failed_body = item_event ++ + "data: {\"type\":\"response.failed\",\"response\":{\"error\":{\"message\":\"summary rejected\",\"code\":\"context_length_exceeded\"}}}\n"; + switch (try parseResponses(&agent, failed_body)) { + .err => |failure| { + try std.testing.expectEqualStrings("summary rejected", failure.message); + try std.testing.expectEqualStrings("context_length_exceeded", failure.code.?); + }, + .ok => return error.TestUnexpectedResult, + } + + const incomplete_body = item_event ++ + "data: {\"type\":\"response.incomplete\",\"response\":{\"usage\":{\"total_tokens\":42}}}\n"; + switch (try parseResponses(&agent, incomplete_body)) { + .ok => |root| { + try std.testing.expectEqual(@as(usize, 1), root.get("output").?.array.items.len); + try std.testing.expect(root.get("incomplete").?.bool); + }, + .err => return error.TestUnexpectedResult, + } + + switch (try parseResponses(&agent, item_event)) { + .ok => |root| try std.testing.expect(root.get("incomplete").?.bool), + .err => return error.TestUnexpectedResult, + } + + switch (try parseResponses(&agent, "{\"error\":{\"code\":\"context_length_exceeded\"}}")) { + .err => |failure| { + try std.testing.expectEqualStrings("context_length_exceeded", failure.code.?); + try std.testing.expect(failure.message.len > 0); + }, + .ok => return error.TestUnexpectedResult, + } +} + +pub fn errorMessage(obj: std.json.ObjectMap) ?[]const u8 { + if (obj.get("error")) |e| { + if (e == .object) { + if (e.object.get("message")) |m| if (m == .string) return m.string; + } else if (e == .string) return e.string; + } + if (obj.get("response")) |r| if (r == .object) { + if (r.object.get("error")) |e| if (e == .object) { + if (e.object.get("message")) |m| if (m == .string) return m.string; + }; + }; + if (obj.get("detail")) |d| if (d == .string) return d.string; + if (obj.get("message")) |m| if (m == .string) return m.string; + return null; +} diff --git a/src/agent_steps.zig b/src/agent_steps.zig index eee3e0c3..91eb5dfe 100644 --- a/src/agent_steps.zig +++ b/src/agent_steps.zig @@ -21,6 +21,13 @@ const messages_mod = @import("messages.zig"); const util = @import("util.zig"); const toolResultMessage = messages_mod.toolResultMessage; +fn surfaceUnstreamedText(self: *Agent, text: []const u8) !void { + if (self.sub or self.streamed_text or text.len == 0) return; + if (main_mod.json_mode) { + self.emit(.{ .type = "text", .text = text }); + } else try self.say("{s}\n", .{text}); +} + // ssePayload/sseIndex live in agent_stream.zig; reached through the Agent // struct's member aliases. const ssePayload = Agent.ssePayload; @@ -50,7 +57,7 @@ pub fn stepResponses(self: *Agent, response: std.json.ObjectMap) !?[]const u8 { if (std.mem.eql(u8, bt, "output_text")) { if (block.object.get("text")) |txt| if (txt == .string) { final_text = txt.string; - if (!self.sub and !self.streamed_text) try self.say("{s}\n", .{txt.string}); + try surfaceUnstreamedText(self, txt.string); }; } } @@ -66,6 +73,7 @@ pub fn stepResponses(self: *Agent, response: std.json.ObjectMap) !?[]const u8 { try calls.append(self.gpa, .{ .id = call_id, .name = name, .input = input }); } } + self.pairContextMeterWithCurrentLocal(); // Codex WS delta: the server now holds up to here (the output items appended // above). The NEXT request sends previous_response_id + only what is appended @@ -115,7 +123,7 @@ pub fn stepAnthropic(self: *Agent, root: std.json.ObjectMap) !?[]const u8 { if (std.mem.eql(u8, kind, "text")) { if (block.object.get("text")) |tx| if (tx == .string) { final_text = tx.string; - if (!self.sub and !self.streamed_text) try self.say("{s}\n", .{final_text}); + try surfaceUnstreamedText(self, final_text); }; } else if (std.mem.eql(u8, kind, "tool_use")) { const name = if (block.object.get("name")) |n| (if (n == .string) n.string else "") else ""; @@ -125,6 +133,7 @@ pub fn stepAnthropic(self: *Agent, root: std.json.ObjectMap) !?[]const u8 { try calls.append(self.gpa, .{ .id = id, .name = name, .input = input }); } } + self.pairContextMeterWithCurrentLocal(); if (calls.items.len > 0) { const results = try self.runTools(calls.items); @@ -163,7 +172,7 @@ pub fn stepOpenAI(self: *Agent, root: std.json.ObjectMap) !?[]const u8 { var final_text: []const u8 = ""; if (message.object.get("content")) |c| if (c == .string and c.string.len > 0) { final_text = c.string; - if (!self.sub and !self.streamed_text) try self.say("{s}\n", .{c.string}); + try surfaceUnstreamedText(self, c.string); }; var calls: std.ArrayList(ToolCall) = .empty; @@ -184,6 +193,7 @@ pub fn stepOpenAI(self: *Agent, root: std.json.ObjectMap) !?[]const u8 { try calls.append(self.gpa, .{ .id = id, .name = name, .input = input }); } }; + self.pairContextMeterWithCurrentLocal(); if (calls.items.len > 0) { const results = try self.runTools(calls.items); @@ -239,6 +249,7 @@ pub fn assembleAnthropic(self: *Agent, body: []const u8) !?std.json.ObjectMap { // the next request's scratch reset). Previously every event's parse tree // landed on the session arena for the life of the process. const scratch = self.scratchAlloc(); + const result_arena = self.messageMutationAlloc(); var root: ?std.json.ObjectMap = null; var blocks: std.ArrayList(BlockAcc) = .empty; var stop_reason: ?Value = null; @@ -284,7 +295,7 @@ pub fn assembleAnthropic(self: *Agent, body: []const u8) !?std.json.ObjectMap { // type=="error" check reports it. Detached from scratch — the // rebuild loop can reset the scratch arena before the message // is done being read. - return (try util.dupeJsonValue(self.arena, v)).object; + return (try util.dupeJsonValue(result_arena, v)).object; } } var r = root orelse return null; @@ -302,6 +313,7 @@ pub fn assembleAnthropic(self: *Agent, body: []const u8) !?std.json.ObjectMap { } try r.put(scratch, "content", .{ .array = content }); try r.put(scratch, "stop_reason", stop_reason orelse Value{ .string = "end_turn" }); + if (stop_reason == null) try r.put(scratch, "incomplete", .{ .bool = true }); if (usage_delta) |ud| { var usage: std.json.ObjectMap = .empty; if (r.get("usage")) |base| if (base == .object) { @@ -312,7 +324,7 @@ pub fn assembleAnthropic(self: *Agent, body: []const u8) !?std.json.ObjectMap { while (e.next()) |kv| try usage.put(scratch, kv.key_ptr.*, kv.value_ptr.*); try r.put(scratch, "usage", .{ .object = usage }); } - return (try util.dupeJsonValue(self.arena, .{ .object = r })).object; + return (try util.dupeJsonValue(result_arena, .{ .object = r })).object; } /// One in-flight tool call while reassembling an OpenAI chat stream; @@ -325,6 +337,7 @@ const CallAcc = struct { }; pub fn assembleOpenAI(self: *Agent, body: []const u8) !?std.json.ObjectMap { + const result_arena = self.messageMutationAlloc(); var content: std.ArrayList(u8) = .empty; var reasoning_content: std.ArrayList(u8) = .empty; var reasoning: std.ArrayList(u8) = .empty; @@ -357,9 +370,9 @@ pub fn assembleOpenAI(self: *Agent, body: []const u8) !?std.json.ObjectMap { if (d.object.get("role")) |x| if (x == .string) { role = x.string; }; - if (d.object.get("content")) |x| if (x == .string) try content.appendSlice(self.arena, x.string); - if (d.object.get("reasoning_content")) |x| if (x == .string) try reasoning_content.appendSlice(self.arena, x.string); - if (d.object.get("reasoning")) |x| if (x == .string) try reasoning.appendSlice(self.arena, x.string); + if (d.object.get("content")) |x| if (x == .string) try content.appendSlice(result_arena, x.string); + if (d.object.get("reasoning_content")) |x| if (x == .string) try reasoning_content.appendSlice(result_arena, x.string); + if (d.object.get("reasoning")) |x| if (x == .string) try reasoning.appendSlice(result_arena, x.string); if (d.object.get("tool_calls")) |tcs| if (tcs == .array) { for (tcs.array.items) |tc| { if (tc != .object) continue; @@ -369,7 +382,7 @@ pub fn assembleOpenAI(self: *Agent, body: []const u8) !?std.json.ObjectMap { // the fragment continues the latest one. break :blk if (tc.object.get("id") != null or calls.items.len == 0) calls.items.len else calls.items.len - 1; }; - while (calls.items.len <= idx) try calls.append(self.arena, .{}); + while (calls.items.len <= idx) try calls.append(result_arena, .{}); const acc = &calls.items[idx]; if (tc.object.get("id")) |x| if (x == .string and x.string.len > 0) { acc.id = x.string; @@ -378,7 +391,7 @@ pub fn assembleOpenAI(self: *Agent, body: []const u8) !?std.json.ObjectMap { if (f.object.get("name")) |x| if (x == .string and x.string.len > 0) { acc.name = x.string; }; - if (f.object.get("arguments")) |x| if (x == .string) try acc.args.appendSlice(self.arena, x.string); + if (f.object.get("arguments")) |x| if (x == .string) try acc.args.appendSlice(result_arena, x.string); }; } }; @@ -386,33 +399,34 @@ pub fn assembleOpenAI(self: *Agent, body: []const u8) !?std.json.ObjectMap { if (!saw_chunk) return null; var message: std.json.ObjectMap = .empty; - try message.put(self.arena, "role", .{ .string = try self.arena.dupe(u8, role) }); // #124: role slices the scratch parse tree; dupe so it survives the per-request scratch reset - try message.put(self.arena, "content", if (content.items.len > 0) Value{ .string = content.items } else .null); - if (reasoning_content.items.len > 0) try message.put(self.arena, "reasoning_content", .{ .string = reasoning_content.items }); - if (reasoning.items.len > 0) try message.put(self.arena, "reasoning", .{ .string = reasoning.items }); + try message.put(result_arena, "role", .{ .string = try result_arena.dupe(u8, role) }); // #124: role slices the scratch parse tree; dupe so it survives the per-request scratch reset + try message.put(result_arena, "content", if (content.items.len > 0) Value{ .string = content.items } else .null); + if (reasoning_content.items.len > 0) try message.put(result_arena, "reasoning_content", .{ .string = reasoning_content.items }); + if (reasoning.items.len > 0) try message.put(result_arena, "reasoning", .{ .string = reasoning.items }); if (calls.items.len > 0) { - var tcs = std.json.Array.init(self.arena); + var tcs = std.json.Array.init(result_arena); for (calls.items) |c| { if (c.id.len == 0 and c.name.len == 0) continue; var function: std.json.ObjectMap = .empty; - try function.put(self.arena, "name", .{ .string = try self.arena.dupe(u8, c.name) }); // #124: dupe off the scratch parse tree - try function.put(self.arena, "arguments", .{ .string = c.args.items }); + try function.put(result_arena, "name", .{ .string = try result_arena.dupe(u8, c.name) }); // #124: dupe off the scratch parse tree + try function.put(result_arena, "arguments", .{ .string = c.args.items }); var tc: std.json.ObjectMap = .empty; - try tc.put(self.arena, "id", .{ .string = try self.arena.dupe(u8, c.id) }); // #124: dupe off the scratch parse tree - try tc.put(self.arena, "type", .{ .string = "function" }); - try tc.put(self.arena, "function", .{ .object = function }); + try tc.put(result_arena, "id", .{ .string = try result_arena.dupe(u8, c.id) }); // #124: dupe off the scratch parse tree + try tc.put(result_arena, "type", .{ .string = "function" }); + try tc.put(result_arena, "function", .{ .object = function }); try tcs.append(.{ .object = tc }); } - if (tcs.items.len > 0) try message.put(self.arena, "tool_calls", .{ .array = tcs }); + if (tcs.items.len > 0) try message.put(result_arena, "tool_calls", .{ .array = tcs }); } var choice: std.json.ObjectMap = .empty; - try choice.put(self.arena, "message", .{ .object = message }); - try choice.put(self.arena, "finish_reason", finish orelse Value{ .string = "stop" }); - var choices = std.json.Array.init(self.arena); + try choice.put(result_arena, "message", .{ .object = message }); + try choice.put(result_arena, "finish_reason", finish orelse Value{ .string = "stop" }); + var choices = std.json.Array.init(result_arena); try choices.append(.{ .object = choice }); var r: std.json.ObjectMap = .empty; - try r.put(self.arena, "choices", .{ .array = choices }); - if (usage) |u| try r.put(self.arena, "usage", u); + try r.put(result_arena, "choices", .{ .array = choices }); + if (usage) |u| try r.put(result_arena, "usage", u); + if (finish == null) try r.put(result_arena, "incomplete", .{ .bool = true }); return r; } diff --git a/src/agent_ws_test.zig b/src/agent_ws_test.zig new file mode 100644 index 00000000..17c13213 --- /dev/null +++ b/src/agent_ws_test.zig @@ -0,0 +1,103 @@ +//! Regression tests for Codex WebSocket delta-session re-anchoring. + +const std = @import("std"); +const Agent = @import("agent.zig").Agent; +const textMessage = @import("messages.zig").textMessage; +const ws = @import("ws.zig"); +const agent_ws = @import("agent_ws.zig"); + +test "closeCodexWs resets the delta session state + frees the response id (codex-ws)" { + var agent: Agent = undefined; + agent.gpa = std.testing.allocator; + agent.codex_ws = null; // no live WsClient to deinit in a unit test + agent.codex_prev_id = try std.testing.allocator.dupe(u8, "resp_abc123"); // must be freed (leak-checked) + agent.codex_sent_upto = 5; + agent.closeCodexWs(); + try std.testing.expect(agent.codex_prev_id == null); // freed + nulled + try std.testing.expectEqual(@as(usize, 0), agent.codex_sent_upto); // delta boundary reset + try std.testing.expect(agent.codex_ws == null); +} + +// (#codex-ws) The delta-body detection string check in agent_ws.zig's +// postLive() gates the WS-reanchor path (never SSE-replay a delta): it +// looks for the literal `"previous_response_id"` key that buildBody's +// .responses branch emits. Pin the exact substring so the two stay in +// sync — a future rename of the JSON key on either side breaks this test +// instead of silently reopening the SSE-replay bug. +test "postLive's delta-body detection matches the key buildBody emits (codex-ws)" { + const delta_body = "{\"model\":\"gpt-5\",\"previous_response_id\":\"resp_1\",\"input\":[]}"; + const full_body = "{\"model\":\"gpt-5\",\"input\":[]}"; + try std.testing.expect(std.mem.indexOf(u8, delta_body, "\"previous_response_id\"") != null); + try std.testing.expect(std.mem.indexOf(u8, full_body, "\"previous_response_id\"") == null); +} + +// (#codex-ws) The preemptive idle re-anchor decision (postResponsesWs closes a +// held WS the server has likely already killed instead of eating a failed +// round trip). Pure helper so no socket is needed: exactly-at-limit must NOT +// expire (only strictly past it), and a WS used moments ago must survive. +// opencode pools at 5 min; ours defaults to 4 (the backend killed a real +// session within 8.5 min idle). +test "codexWsIdleExpired: fires only strictly past the idle limit (codex-ws)" { + const limit = agent_ws.codex_ws_idle_ms; + try std.testing.expectEqual(@as(i64, 4 * std.time.ms_per_min), limit); // default: stay under the observed server kill + try std.testing.expect(!agent_ws.codexWsIdleExpired(1_000_000, 1_000_000)); // just used + try std.testing.expect(!agent_ws.codexWsIdleExpired(1_000_000 + limit, 1_000_000)); // exactly at the limit — keep + try std.testing.expect(agent_ws.codexWsIdleExpired(1_000_000 + limit + 1, 1_000_000)); // past it — re-anchor + try std.testing.expect(agent_ws.codexWsIdleExpired(1_000_000 + 510 * std.time.ms_per_s, 1_000_000)); // the real 8.5-min trace gap +} + +// (#codex-ws) End-to-end regression for the reanchor fix: buildBody's +// .responses branch must emit previous_response_id + a message-slice delta +// while a WS session + prev id are held, and after closeCodexWs (called by +// postLive on a delta transport error, and again by request()'s +// error.CodexWsReanchor handler) a rebuilt body must carry the FULL +// message history with no previous_response_id — never a stale delta +// replayed against a dead session. +test "buildBody (.responses): delta while WS live; full input after closeCodexWs (codex-ws)" { + var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator); + defer arena_state.deinit(); + const a = arena_state.allocator(); + + var msgs = std.json.Array.init(a); + try msgs.append(try textMessage(a, "user", "first")); + try msgs.append(try textMessage(a, "assistant", "second")); + try msgs.append(try textMessage(a, "user", "third — not yet sent")); + + var dummy_ws: ws.WsClient = undefined; // buildBody only checks != null, never dereferences + var agent: Agent = .{ + .gpa = std.testing.allocator, + .arena = a, + .io = undefined, // unused by buildBody + .client = undefined, // unused by buildBody + .provider = .{ .id = "codex", .kind = .responses, .auth = .bearer, .url = "https://x/responses", .api_key = "k", .model = "gpt-5", .context = 100_000 }, + .messages = msgs, + .sub = false, + .label = "", + .out = null, + .codex_ws = &dummy_ws, + .codex_prev_id = try std.testing.allocator.dupe(u8, "resp_live"), + .codex_sent_upto = 2, // server already holds messages[0..2]; delta = [2..] + }; + + const delta_body = try agent.buildBody(null, false, false, false); + defer std.testing.allocator.free(delta_body); + try std.testing.expect(std.mem.indexOf(u8, delta_body, "\"previous_response_id\":\"resp_live\"") != null); + try std.testing.expect(std.mem.indexOf(u8, delta_body, "third — not yet sent") != null); + try std.testing.expect(std.mem.indexOf(u8, delta_body, "\"first\"") == null); // NOT resent — already on the server + + // Simulate closeCodexWs's effect (covered by its own unit test above) + // without invoking it directly: it would call ws.WsClient.deinit on + // codex_ws, which sends a real close frame over `dummy_ws`'s + // uninitialized io/fd — fine for a live connection, unsafe for this + // struct-literal stand-in. What matters here is buildBody's behavior + // given the post-close state postLive/request() leave it in. + std.testing.allocator.free(agent.codex_prev_id.?); + agent.codex_ws = null; + agent.codex_prev_id = null; + agent.codex_sent_upto = 0; + const rebuilt_body = try agent.buildBody(null, false, false, false); + defer std.testing.allocator.free(rebuilt_body); + try std.testing.expect(std.mem.indexOf(u8, rebuilt_body, "\"previous_response_id\"") == null); + try std.testing.expect(std.mem.indexOf(u8, rebuilt_body, "\"first\"") != null); // full history restored + try std.testing.expect(std.mem.indexOf(u8, rebuilt_body, "third — not yet sent") != null); +} diff --git a/src/commands_misc.zig b/src/commands_misc.zig index 6024efa8..efa9b155 100644 --- a/src/commands_misc.zig +++ b/src/commands_misc.zig @@ -37,11 +37,6 @@ const models_cache = @import("models_cache.zig"); const command_catalog = @import("command_catalog.zig"); const serde = @import("serde.zig"); -const schema = @import("schema.zig"); -const renderRootTools = schema.renderRootTools; -const root_specs = schema.root_specs; -const effectiveRootSpecs = schema.effectiveRootSpecs; // #225: gate clock_sleep on the flag when re-rendering the tool lists - const mcp_cli = @import("mcp_cli.zig"); const persistMcpServer = mcp_cli.persistMcpServer; @@ -67,6 +62,8 @@ fn providerModelCount(provider_id: []const u8) usize { } fn showModelsHealth(root: *Agent, keys: *Keys, arena: Allocator, out: *Io.Writer) !void { + root.ensureStoredKeys(keys); + root.ensureModelCatalog(keys.*); const saved = serde.loadModel(root.io, arena, root.home); try out.print("{s}model health{s}\n", .{ style.bold, style.reset }); try out.print("active: {s} via {s} · {d}k ctx · compact@{d}k{s}{s}\n", .{ @@ -205,11 +202,11 @@ pub fn tryHandle(root: *Agent, keys: *Keys, arena: Allocator, line: []const u8, try out.flush(); return true; }; - // Re-render the tool lists so the new tools reach the model. - const root_tool_specs = try effectiveRootSpecs(arena); // #225: gate clock_sleep on the flag - root.tools_anthropic = try renderRootTools(arena, .anthropic, root_tool_specs, reg.tools); - root.tools_openai = try renderRootTools(arena, .openai, root_tool_specs, reg.tools); - root.tools_responses = try renderRootTools(arena, .responses, root_tool_specs, reg.tools); + // Re-render the active catalog so the new tools reach the model; + // inactive provider formats stay lazy until a later switch. + root.invalidateRootTools(); + try root.ensureRootTools(root.provider.kind); + root.rebaseContextMeter(); const persisted = persistMcpServer(root.io, arena, name, command, args.items); var has_note = false; for (mcp_notes) |mn| if (std.mem.eql(u8, mn.server, name)) { @@ -234,11 +231,10 @@ pub fn tryHandle(root: *Agent, keys: *Keys, arena: Allocator, line: []const u8, if (n == 0) { try out.writeAll("no untrusted workspace MCP server(s) to connect.\n"); } else { - // Re-render the tool lists so the new tools reach the model. - const root_tool_specs = try effectiveRootSpecs(arena); // #225: gate clock_sleep on the flag - root.tools_anthropic = try renderRootTools(arena, .anthropic, root_tool_specs, reg.tools); - root.tools_openai = try renderRootTools(arena, .openai, root_tool_specs, reg.tools); - root.tools_responses = try renderRootTools(arena, .responses, root_tool_specs, reg.tools); + // Re-render the active catalog; other wire formats remain lazy. + root.invalidateRootTools(); + try root.ensureRootTools(root.provider.kind); + root.rebaseContextMeter(); try out.print("{s}✓{s} trusted workspace — connected {d} MCP server(s); {d} tool(s) total\n", .{ style.green, style.reset, n, reg.tools.len }); } try out.flush(); @@ -265,6 +261,8 @@ pub fn tryHandle(root: *Agent, keys: *Keys, arena: Allocator, line: []const u8, return true; } if (std.mem.eql(u8, line, "/models")) { + root.ensureStoredKeys(keys); + root.ensureModelCatalog(keys.*); try out.writeAll("model ctx compact@ provider key vision\n"); for (pricing.models()) |m| { const has_key = keys.get(m.provider) != null; @@ -392,6 +390,7 @@ pub fn tryHandle(root: *Agent, keys: *Keys, arena: Allocator, line: []const u8, return true; } if (std.mem.startsWith(u8, line, "/resume")) { + root.ensureStoredKeys(keys); const arg = std.mem.trim(u8, line["/resume".len..], " \t"); var name: []const u8 = if (arg.len == 0) "last" else arg; // Bare /resume on a TTY: pick from the saved sessions interactively, @@ -419,7 +418,7 @@ pub fn tryHandle(root: *Agent, keys: *Keys, arena: Allocator, line: []const u8, const idx = listPicker(root, arena, out, "Resume session ›", sessions.items) orelse return true; name = entries.items[idx].base; } - loadSession(root, keys.*, arena, name) catch |err| { + loadSession(root, keys, arena, name) catch |err| { switch (err) { error.FileNotFound => try out.print("no session named '{s}' ({s}{s} not found in cwd) — /sessions lists saved ones\n", .{ name, name, session_ext }), else => try out.print("resume failed: {t}\n", .{err}), diff --git a/src/commands_model.zig b/src/commands_model.zig index 72323b7d..28bd0023 100644 --- a/src/commands_model.zig +++ b/src/commands_model.zig @@ -124,13 +124,15 @@ fn handleFallback(root: *Agent, arena: Allocator, line: []const u8, out: *Io.Wri pub fn tryHandle(root: *Agent, keys: *Keys, arena: Allocator, line: []const u8, out: *Io.Writer) !bool { if (try handleFallback(root, arena, line, out)) return true; if (std.mem.startsWith(u8, line, "/model") and !std.mem.startsWith(u8, line, "/models")) { + root.ensureStoredKeys(keys); const arg = std.mem.trim(u8, line["/model".len..], " \t"); + if (providers.modelQueryMayUseCodex(arg)) root.ensureModelCatalog(keys.*); if (arg.len == 0) { if (main_mod.use_color) { // interactive TTY → fuzzy picker if (modelPicker(root, keys, arena, out)) |idx| { const m = pricing.models()[idx]; const provider = keys.providerById(m.provider, m.name) catch { - try offerProviderAuth(root, keys, arena, out, m.provider, m.name); + try offerProviderAuth(root, keys, arena, out, m.provider, m.name, false); return true; }; try switchProvider(root, arena, provider, out); @@ -168,7 +170,7 @@ pub fn tryHandle(root: *Agent, keys: *Keys, arena: Allocator, line: []const u8, } const m = try arena.dupe(u8, mdl); const provider = keys.providerById(pid, m) catch { - try offerProviderAuth(root, keys, arena, out, pid, m); + try offerProviderAuth(root, keys, arena, out, pid, m, false); return true; }; try switchProvider(root, arena, provider, out); @@ -184,7 +186,7 @@ pub fn tryHandle(root: *Agent, keys: *Keys, arena: Allocator, line: []const u8, // baked default. One loaded → switch straight to it; many → list to pick. if (isLocalUrl(spec.url)) { const key = keys.get(spec.id) orelse { - try offerProviderAuth(root, keys, arena, out, spec.id, pricing.providerDefaultModel(spec.id, spec.default_model)); + try offerProviderAuth(root, keys, arena, out, spec.id, pricing.providerDefaultModel(spec.id, spec.default_model), true); return true; }; const murl = openAiModelsUrl(arena, spec.url); @@ -205,7 +207,7 @@ pub fn tryHandle(root: *Agent, keys: *Keys, arena: Allocator, line: []const u8, } const default_model = pricing.providerDefaultModel(spec.id, spec.default_model); const provider = keys.providerById(spec.id, default_model) catch { - try offerProviderAuth(root, keys, arena, out, spec.id, default_model); + try offerProviderAuth(root, keys, arena, out, spec.id, default_model, true); return true; }; try switchProvider(root, arena, provider, out); @@ -219,7 +221,7 @@ pub fn tryHandle(root: *Agent, keys: *Keys, arena: Allocator, line: []const u8, const name = try arena.dupe(u8, resolved); const provider = keys.providerFor(name) catch { for (pricing.models()) |mt| if (std.mem.eql(u8, mt.name, name)) { - try offerProviderAuth(root, keys, arena, out, mt.provider, name); + try offerProviderAuth(root, keys, arena, out, mt.provider, name, false); return true; }; try out.writeAll("no API key for any provider serving that model — see /models, or add one with /key \n"); @@ -231,7 +233,7 @@ pub fn tryHandle(root: *Agent, keys: *Keys, arena: Allocator, line: []const u8, } if (std.mem.eql(u8, line, "/compact")) { _ = root.compact() catch |err| switch (err) { - error.ApiError => {}, + error.ApiError, error.EmptySummary, error.IncompleteSummary => {}, else => |e| return e, }; return true; @@ -277,6 +279,8 @@ pub fn tryHandle(root: *Agent, keys: *Keys, arena: Allocator, line: []const u8, const dropped = root.messages.items.len - cut; root.messages.items.len = cut; // truncate (entries are arena-owned) root.last_context_tokens = 0; + root.context_local_tokens = 0; + root.compact_transport_failures = 0; // Restore files written/edited during the rewound turns, and re-point the // turn counter so the next prompt re-takes turn n. var restored: usize = 0; @@ -395,6 +399,7 @@ pub fn tryHandle(root: *Agent, keys: *Keys, arena: Allocator, line: []const u8, return true; } if (std.mem.startsWith(u8, line, "/key")) { + root.ensureStoredKeys(keys); const rest = std.mem.trim(u8, line["/key".len..], " \t"); if (rest.len == 0) { // show key status + how to add try out.writeAll("API keys (✓ = set via env / Keychain / login):\n"); @@ -435,6 +440,7 @@ pub fn tryHandle(root: *Agent, keys: *Keys, arena: Allocator, line: []const u8, return true; } if (std.mem.startsWith(u8, line, "/login")) { + root.ensureStoredKeys(keys); // Interactive OAuth sign-in for the providers that have a device/PKCE // flow (codegraff, codex/ChatGPT, kimi). Mirrors the `graff login` // subcommands but runs in-session and pulls the fresh key into the live @@ -565,6 +571,7 @@ pub fn tryHandle(root: *Agent, keys: *Keys, arena: Allocator, line: []const u8, } if (std.mem.eql(u8, line, "/strict")) { root.strict = !root.strict; + root.rebaseContextMeter(); try out.print("strict mode {s} — {s}\n", .{ if (root.strict) "ON" else "off", if (root.strict) "every message must be a tool; finish with attempt_completion" else "free-text replies allowed", diff --git a/src/commands_session.zig b/src/commands_session.zig index d24192a7..aa827033 100644 --- a/src/commands_session.zig +++ b/src/commands_session.zig @@ -78,7 +78,9 @@ pub fn tryHandle(root: *Agent, keys: *Keys, arena: Allocator, line: []const u8, if (std.mem.eql(u8, line, "/clear")) { root.messages = std.json.Array.init(arena); root.last_context_tokens = 0; + root.context_local_tokens = 0; root.last_cache_read = 0; + root.compact_transport_failures = 0; root.tui_header_shown = false; root.session_title = null; // re-summarize the now-empty conversation root.ai_title_done = false; @@ -95,7 +97,9 @@ pub fn tryHandle(root: *Agent, keys: *Keys, arena: Allocator, line: []const u8, if (std.mem.eql(u8, line, "/new")) { root.messages = std.json.Array.init(arena); root.last_context_tokens = 0; + root.context_local_tokens = 0; root.last_cache_read = 0; + root.compact_transport_failures = 0; root.todos.clearRetainingCapacity(); resetConversationSteering(root); root.session_title = null; diff --git a/src/fleet.zig b/src/fleet.zig index 318badfb..84bcdb79 100644 --- a/src/fleet.zig +++ b/src/fleet.zig @@ -20,6 +20,7 @@ const root = @import("main.zig"); const trace = @import("trace.zig"); const telemetry_mod = @import("telemetry.zig"); const Telemetry = telemetry_mod.Telemetry; +const http = @import("http.zig"); const trajectory_path = trace.trajectory_path; // ── Agent types (the MAP-Elites niches) ───────────────────────────────────── @@ -263,6 +264,7 @@ pub fn agentTypePrompt(name: []const u8) ?[]const u8 { /// live beside it at /v1/elites. pub fn pullElites(io: Io, arena: Allocator, client: *std.http.Client, telem: ?*Telemetry, endpoint: []const u8, provider_class: []const u8, eval_set_hash: []const u8, types: []const AgentType) []const AgentType { if (endpoint.len == 0 or !root.g_fleet) return types; + http.waitForClientReady(io); var base = std.mem.trimEnd(u8, endpoint, "/"); if (std.mem.endsWith(u8, base, "/v1/logs")) base = base[0 .. base.len - "/v1/logs".len]; // No eval suite → omit eval_set_hash entirely (NOT empty): the worker reads an diff --git a/src/http.zig b/src/http.zig index d77ffa95..c6fc196c 100644 --- a/src/http.zig +++ b/src/http.zig @@ -17,6 +17,14 @@ const Agent = agent_mod.Agent; const anthropic_version = root.anthropic_version; const kimi_user_agent = root.kimi_user_agent; +/// Launch-scoped gate installed while the shared client's CA bundle warms in +/// the background. Null in unit tests and standalone pre-client subcommands. +pub var g_client_ready: ?*Io.Event = null; + +pub fn waitForClientReady(io: Io) void { + if (g_client_ready) |ready| ready.waitUncancelable(io); +} + pub fn providerUserAgent(provider: Provider) std.http.Client.Request.Headers.Value { if (std.mem.eql(u8, provider.id, "kimi")) { return .{ .override = kimi_user_agent }; diff --git a/src/keys_cli.zig b/src/keys_cli.zig index 4a32a73e..ec76a2e3 100644 --- a/src/keys_cli.zig +++ b/src/keys_cli.zig @@ -5,7 +5,8 @@ //! "simple-harness", account=provider id) via the `security` CLI — never on //! disk in plaintext. Elsewhere they fall back to a 0600 file //! (~/.simple-harness-keys.json). `harness key set ` writes; -//! startup reads for any provider whose env var isn't set (env always wins). +//! startup reads the selected provider first (env always wins), then fills the +//! remaining providers when a picker, switch, resume, or fallback needs them. //! //! storeKey stays pub — commands_model.zig and pickers.zig back-import it as //! `main_mod.storeKey`; isLocalUrl/openAiModelsUrl/fetchOpenAIModels stay pub @@ -153,6 +154,75 @@ pub fn loadStoredKey(io: Io, arena: Allocator, home: []const u8, provider: []con return null; } +pub const StoredKeyScope = union(enum) { + all, + provider: []const u8, + mask: [provider_specs.len]bool, + + pub fn includes(scope: StoredKeyScope, index: usize, provider_id: []const u8) bool { + return switch (scope) { + .all => true, + .provider => |id| std.mem.eql(u8, id, provider_id), + .mask => |mask| mask[index], + }; + } +}; + +test "StoredKeyScope selects all, one provider, or an exact mask" { + const all: StoredKeyScope = .all; + try std.testing.expect(all.includes(0, provider_specs[0].id)); + const one: StoredKeyScope = .{ .provider = "deepseek" }; + try std.testing.expect(one.includes(2, "deepseek")); + try std.testing.expect(!one.includes(1, "codegraff")); + var mask = [_]bool{false} ** provider_specs.len; + mask[1] = true; + const selected: StoredKeyScope = .{ .mask = mask }; + try std.testing.expect(selected.includes(1, "codegraff")); + try std.testing.expect(!selected.includes(2, "deepseek")); +} + +fn loadStoredKeyTask(io: Io, gpa: Allocator, home: []const u8, provider_id: []const u8) ?[]u8 { + var task_arena = std.heap.ArenaAllocator.init(gpa); + defer task_arena.deinit(); + const key = loadStoredKey(io, task_arena.allocator(), home, provider_id) orelse return null; + return gpa.dupe(u8, key) catch null; +} + +/// Fill selected missing credential slots. macOS stores one Keychain item per +/// provider, so its independent `security` processes run concurrently with +/// task-local arenas. Other platforms keep all keys in one JSON file: read and +/// parse it once, then fill every selected slot from that single object. +pub fn loadMissingStoredKeys(io: Io, gpa: Allocator, arena: Allocator, home: []const u8, keys: *provider_mod.Keys, scope: StoredKeyScope) void { + if (builtin.os.tag == .macos) { + var futures = [_]?Io.Future(?[]u8){null} ** provider_specs.len; + for (provider_specs, keys.values, 0..) |spec, value, i| { + if (value != null or !scope.includes(i, spec.id)) continue; + const task_args = .{ io, gpa, home, spec.id }; + futures[i] = io.concurrent(loadStoredKeyTask, task_args) catch + io.async(loadStoredKeyTask, task_args); + } + for (&futures, &keys.values, &keys.sources) |*maybe_future, *value, *source| { + const owned = if (maybe_future.*) |*future| future.await(io) orelse continue else continue; + value.* = arena.dupe(u8, owned) catch null; + gpa.free(owned); + if (value.* != null) source.* = .stored; + } + return; + } + + const path = std.fmt.allocPrint(arena, "{s}/{s}", .{ home, keys_file }) catch return; + const data = Io.Dir.cwd().readFileAlloc(io, path, arena, .limited(64 * 1024)) catch return; + const parsed = std.json.parseFromSliceLeaky(Value, arena, data, .{ .allocate = .alloc_always }) catch return; + if (parsed != .object) return; + for (provider_specs, &keys.values, &keys.sources, 0..) |spec, *value, *source, i| { + if (value.* != null or !scope.includes(i, spec.id)) continue; + const stored = parsed.object.get(spec.id) orelse continue; + if (stored != .string or stored.string.len == 0) continue; + value.* = stored.string; + source.* = .stored; + } +} + /// `harness key set ` / `harness key list` — manage the safe /// key store. Validates the provider id against provider_specs. pub fn keyCommand(io: Io, gpa: Allocator, arena: Allocator, home: []const u8, args: []const []const u8) !void { @@ -161,9 +231,11 @@ pub fn keyCommand(io: Io, gpa: Allocator, arena: Allocator, home: []const u8, ar const out = &ow.interface; if (args.len == 0 or std.mem.eql(u8, args[0], "list")) { + var stored_keys: provider_mod.Keys = .{ .values = [_]?[]const u8{null} ** provider_specs.len }; + loadMissingStoredKeys(io, gpa, arena, home, &stored_keys, .all); try out.writeAll("provider env var stored\n"); for (provider_specs) |spec| { - const stored = loadStoredKey(io, arena, home, spec.id) != null; + const stored = stored_keys.get(spec.id) != null; try out.print(" {s:<14}{s:<22}{s}\n", .{ spec.id, spec.env_key, if (stored) "yes" else "—" }); } try out.flush(); diff --git a/src/main.zig b/src/main.zig index f5544e06..5db6a1c2 100644 --- a/src/main.zig +++ b/src/main.zig @@ -19,6 +19,12 @@ const std = @import("std"); const Io = std.Io; +fn bootMark(io: Io, started: Io.Timestamp, enabled: bool, label: []const u8) void { + if (!enabled) return; + const ms = @max(0, started.untilNow(io, .awake).toMilliseconds()); + std.debug.print("[boot +{d}ms] {s}\n", .{ ms, label }); +} + /// #131: pre-load the shared HTTP client's CA bundle single-threaded, so the /// parallel subagents that share this client (on pool threads) never race the /// lazy CA-bundle rescan on their first concurrent HTTPS connect. Zig std flags @@ -31,6 +37,14 @@ fn prewarmCaBundle(client: *std.http.Client, gpa: std.mem.Allocator, io: Io) voi client.ca_bundle.rescan(gpa, io, now) catch return; client.now = now; } + +/// Warm the shared client's CA bundle off the launch critical path. Outbound +/// users wait on `http.g_client_ready`, so the prompt can paint during this +/// scan without reintroducing the concurrent first-connect race above. +fn prewarmCaBundleTask(client: *std.http.Client, gpa: std.mem.Allocator, io: Io, ready: *Io.Event) void { + defer ready.set(io); + prewarmCaBundle(client, gpa, io); +} const Value = std.json.Value; const Allocator = std.mem.Allocator; const mcp = @import("mcp.zig"); @@ -43,6 +57,8 @@ const agent_argstream = @import("agent_argstream.zig"); const agent_render = @import("agent_render.zig"); const agent_steps = @import("agent_steps.zig"); const agent_compact = @import("agent_compact.zig"); +const agent_compact_test = @import("agent_compact_test.zig"); +const agent_ws_test = @import("agent_ws_test.zig"); const agent_tools = @import("agent_tools.zig"); const agent_request = @import("agent_request.zig"); const agent_interrupt = @import("agent_interrupt.zig"); @@ -118,6 +134,8 @@ test { _ = agent_render; _ = agent_steps; _ = agent_compact; + _ = agent_compact_test; + _ = agent_ws_test; _ = agent_tools; _ = agent_request; _ = agent_interrupt; @@ -247,6 +265,8 @@ const startup = @import("startup.zig"); const session_start = @import("session_start.zig"); const session_run = @import("session_run.zig"); pub fn main(init: std.process.Init) !void { + const boot_started = Io.Timestamp.now(init.io, .awake); + const boot_debug = init.environ_map.get("GRAFF_BOOT_DEBUG") != null; const gpa = init.gpa; const io = init.io; const arena = init.arena.allocator(); @@ -265,6 +285,7 @@ pub fn main(init: std.process.Init) !void { if (builtin.os.tag == .windows) tty.enableVtOutput(); // CLI flags: the Flags struct + parsing loop live in args.zig; downstream code reads flags. in place of ~27 locals this block used to declare. const flags = try args.parse(init); + bootMark(init.io, boot_started, boot_debug, "args"); // GRAFF_CODEX_URL: override the codex responses endpoint (localhost mocks / integration tests). Parsed BEFORE subcommand dispatch and // resolveKeys — `graff models [refresh]` fetches the catalog inside runSubcommand and the initial Keys.build runs inside resolveKeys. // environ_map slices are process-lifetime (resolveKeys stores them into Keys the same way), so the value is kept as-is without duping. @@ -276,6 +297,7 @@ pub fn main(init: std.process.Init) !void { // Credential/model resolution (env vars → on-disk logins → `harness key set` store, env always wins; then --model or the last-saved model) lives // in startup.zig as resolveKeys() — pure over env/disk/arena, safe to call outside main()'s own stack frame. const resolved_keys = try startup.resolveKeys(io, gpa, arena, init.environ_map, flags.model_flag); + bootMark(io, boot_started, boot_debug, "credentials/model"); var keys = resolved_keys.keys; const default_provider = resolved_keys.default_provider; const stale_saved_model = resolved_keys.stale_saved_model; @@ -283,7 +305,14 @@ pub fn main(init: std.process.Init) !void { const codex_account = resolved_keys.codex_account; var client: std.http.Client = .{ .allocator = gpa, .io = io }; defer client.deinit(); - prewarmCaBundle(&client, gpa, io); + var client_ready: Io.Event = .unset; + http.g_client_ready = &client_ready; + var client_warm_fut = io.async(prewarmCaBundleTask, .{ &client, gpa, io, &client_ready }); + bootMark(io, boot_started, boot_debug, "CA warm scheduled"); + defer { + _ = client_warm_fut.await(io); + http.g_client_ready = null; + } var stdin_buf: [64 * 1024]u8 = undefined; var stdin_reader = Io.File.stdin().reader(io, &stdin_buf); const in = &stdin_reader.interface; @@ -295,6 +324,7 @@ pub fn main(init: std.process.Init) !void { if (try session_start.runTitleCommand(io, gpa, arena, &client, default_provider, out, flags)) return; try session_start.setupWorktreeAndBanner(io, gpa, arena, init.environ_map, flags, out, trace_path, codex_account, stale_saved_model, preferred_provider, default_provider); + bootMark(io, boot_started, boot_debug, "banner"); // Session trace (best-effort: a failed open just disables tracing). var trace_buf: [8 * 1024]u8 = undefined; @@ -344,6 +374,7 @@ pub fn main(init: std.process.Init) !void { // auto-connect only with --yolo (trusted) or explicit per-session consent; otherwise start with an empty (but live) registry so `/mcp add` // still works. var registry_storage = try session_start.initRegistryConsent(io, gpa, arena, out, in, flags, mcp_config_path, use_color, json_mode); + bootMark(io, boot_started, boot_debug, "MCP registry"); defer registry_storage.deinit(); const registry: ?*mcp.Registry = ®istry_storage; // Per-skill/companion opt-outs, animation/theme settings, and the --selftest-spinner headless render live in session_start.zig. The theme/ @@ -358,13 +389,16 @@ pub fn main(init: std.process.Init) !void { out.flush() catch {}; }; if (theme_setup.should_exit) return; + bootMark(io, boot_started, boot_debug, "settings/theme"); try session_start.connectCompanion(io, ®istry_storage, flags, out, json_mode); const mcp_tools: []const mcp.Tool = registry_storage.tools; // If the metered companion connected, probe its license once so the note below can lean into paid tools (vs the conservative free-codedb note). if (mcpServerConnected(mcp_tools, "codedbpro")) g_codedbpro_licensed = probeCodedbproLicensed(gpa, io); + bootMark(io, boot_started, boot_debug, "companion"); var approvals: Approvals = undefined; try session_run.initApprovalsHooksFleet(io, gpa, arena, init.environ_map, &approvals, flags, out, json_mode); + bootMark(io, boot_started, boot_debug, "approvals/hooks/fleet"); defer { for (approvals.prefixes.items) |p| gpa.free(p); approvals.prefixes.deinit(gpa); @@ -386,6 +420,7 @@ pub fn main(init: std.process.Init) !void { ); const sys_normal = sys_prompt.sys_normal; const sys_strict = sys_prompt.sys_strict; + bootMark(io, boot_started, boot_debug, "system prompt"); var snaps: Snapshots = .{ .gpa = gpa, .io = io }; defer snaps.deinit(); @@ -394,20 +429,23 @@ pub fn main(init: std.process.Init) !void { // Root Agent construction + post-construction config (session name, persisted thinking/goal/eval settings, session-start trace note) + the // backgrounded fleet-champion pull live in session_start.zig. `root`'s pointer fields (snapshots/client/tracer/approvals/registry) all reference // already-stable main()-owned storage passed in by address, so returning the constructed Agent by value here is safe. - var root = try session_run.buildRootAgent(gpa, arena, io, &client, default_provider, init.environ_map, out, in, registry, &approvals, &tracer, sys_normal, sys_strict, mcp_tools, &snaps, flags, telem.endpoint); + var root = try session_run.buildRootAgent(gpa, arena, io, &client, default_provider, init.environ_map, out, in, registry, &approvals, &tracer, sys_normal, sys_strict, &snaps, flags, telem.endpoint); + root.model_catalog = resolved_keys.model_catalog; + root.stored_keys_loaded = resolved_keys.stored_keys_loaded; root.scratch_arena = &scratch_state; // #124: route the root's transient parse garbage here; reset per request() root.fallback_active = stale_saved_model != null; root.fallback_blocked = root.fallback_active and preferred_provider != null and !std.mem.eql(u8, preferred_provider.?, root.provider.id) and !fallback_config.contains(root.fallback_allow, root.provider.id); + bootMark(io, boot_started, boot_debug, "root agent"); defer joinElites(io); // reap if the session quits before any turn joins it - session_run.saveOrResumeSession(&root, keys, arena, flags); + session_run.saveOrResumeSession(&root, &keys, arena, flags); // `graff repl`: interactive chat REPL on the zigzag TUI, backed by the REAL agent loop — each prompt runs a full root turn (tools + MCP) via // replTurnCb, reusing the root agent's tool set + registry + system prompt. Self-contained — exits after. - if (try session_run.runReplCommand(gpa, io, init.environ_map, &root, keys, &client, in, out, arena, flags)) return; + if (try session_run.runReplCommand(gpa, io, init.environ_map, &root, &keys, &client, in, out, arena, flags)) return; // One-shot print mode: run the single prompt to completion, print the final text to stdout, exit. if (flags.oneshot_prompt) |prompt_text| { - try session_run.runOneshotPrompt(gpa, io, arena, &root, keys, &tracer, out, prompt_text); + try session_run.runOneshotPrompt(gpa, io, arena, &root, &keys, &tracer, out, prompt_text); return; } @@ -427,7 +465,7 @@ pub fn main(init: std.process.Init) !void { root.tools_used.deinit(gpa); } const interactive = use_color and !json_mode; // stdout is a TTY → enable line editing - try session_run.restoreResumedSession(io, arena, out, &root, keys, flags, json_mode, g_cwd_display); + try session_run.restoreResumedSession(arena, out, &root, &keys, flags, json_mode, g_cwd_display); // The interactive-REPL / --json-protocol loop lives in mainloop.zig. main() keeps owning every piece of storage the loop touches (root, keys, // tracer/traj/telem via root, history, linebuf, stdin/stdout writers) — Ctx below only holds POINTERS into this stack frame, so nothing dangles diff --git a/src/mainloop.zig b/src/mainloop.zig index a9330f2b..26f6d7f8 100644 --- a/src/mainloop.zig +++ b/src/mainloop.zig @@ -45,6 +45,7 @@ const fleet = @import("fleet.zig"); const jobs = @import("jobs.zig"); const session = @import("session.zig"); const repl_glue = @import("repl_glue.zig"); +const mainloop_score = @import("mainloop_score.zig"); const scoring = @import("scoring.zig"); const trace = @import("trace.zig"); const telemetry = @import("telemetry.zig"); @@ -196,19 +197,24 @@ pub fn run(ctx: *Ctx) !void { // {"type":"score","prompt_sha":"...","score":0.7,"notes":"..."} // appends an evaluation record for an agent variant to the // trajectory archive (the DGM evaluation phase writing back). + var json_request: ?std.json.Parsed(Value) = null; + defer if (json_request) |*request| request.deinit(); const base_msg: []const u8 = if (loop_prompt) |lp| lp else if (goal_prompt) |gp| gp else if (main_mod.json_mode) blk: { - const parsed = std.json.parseFromSliceLeaky(Value, ctx.arena, line, .{ .allocate = .alloc_always }) catch { + json_request = std.json.parseFromSlice(Value, ctx.gpa, line, .{ .allocate = .alloc_if_needed }) catch { ctx.root.emit(.{ .type = "error", .message = "invalid JSON (expect {\"type\":\"user\",\"text\":\"...\"})" }); continue; }; + const parsed = json_request.?.value; const rtype = if (parsed == .object) (if (parsed.object.get("type")) |v| (if (v == .string) v.string else "") else "") else ""; if (std.mem.eql(u8, rtype, "set_model")) { + ctx.root.ensureStoredKeys(ctx.keys); const provider_field = if (parsed.object.get("provider")) |v| (if (v == .string) v.string else "") else ""; const model_field = if (parsed.object.get("model")) |v| (if (v == .string) v.string else "") else ""; const legacy_name = if (parsed.object.get("name")) |v| (if (v == .string) v.string else "") else ""; + if (providers.controlRequestMayUseCodex(provider_field, model_field, legacy_name)) ctx.root.ensureModelCatalog(ctx.keys.*); const provider = providers.resolveProviderControlRequest(ctx.keys, ctx.arena, provider_field, model_field, legacy_name) catch |err| { const label = providers.setModelRequestLabel(ctx.arena, provider_field, model_field, legacy_name) catch ""; const message = switch (err) { @@ -220,7 +226,7 @@ pub fn run(ctx: *Ctx) !void { ctx.root.emit(.{ .type = "error", .message = message }); continue; }; - const note = providers.applyProvider(ctx.root, ctx.arena, provider); + const note = try providers.applyProvider(ctx.root, ctx.arena, provider); ctx.root.emit(.{ .type = "model", .ok = true, .provider = provider.id, .model = provider.model, .context = provider.context, .note = note }); continue; } @@ -228,6 +234,7 @@ pub fn run(ctx: *Ctx) !void { const chars = ctx.root.compact() catch |err| { const message = switch (err) { error.EmptySummary => "compaction failed: empty summary, history unchanged", + error.IncompleteSummary => "compaction failed: incomplete summary, history unchanged", else => try std.fmt.allocPrint(ctx.arena, "compaction failed: {s}", .{@errorName(err)}), }; ctx.root.emit(.{ .type = "error", .message = message }); @@ -254,6 +261,7 @@ pub fn run(ctx: *Ctx) !void { if (id.len == 0) { ctx.root.sys_normal = ctx.sys_normal; ctx.root.sys_strict = ctx.sys_strict; + ctx.root.rebaseContextMeter(); ctx.root.emit(.{ .type = "agent", .ok = true, .id = id, .chars = ctx.root.sys_normal.len }); continue; } @@ -264,6 +272,7 @@ pub fn run(ctx: *Ctx) !void { }; ctx.root.sys_normal = try std.fmt.allocPrint(ctx.arena, "{s}\n\n{s}", .{ ctx.sys_normal, prompt }); ctx.root.sys_strict = try std.fmt.allocPrint(ctx.arena, "{s}{s}", .{ ctx.root.sys_normal, prompts.strict_note }); + ctx.root.rebaseContextMeter(); ctx.root.emit(.{ .type = "agent", .ok = true, .id = id, .chars = ctx.root.sys_normal.len }); continue; } @@ -292,128 +301,7 @@ pub fn run(ctx: *Ctx) !void { continue; } if (std.mem.eql(u8, rtype, "score")) { - // Fingerprint fields must be exactly 16 hex chars — a - // length-only check would let an embedded newline through into - // the signed v2 envelope (field-shift ambiguity). - const hex16 = struct { - fn ok(s: []const u8) bool { - if (s.len != 16) return false; - for (s) |c| if (!std.ascii.isHex(c)) return false; - return true; - } - }; - const sha = if (parsed.object.get("prompt_sha")) |v| (if (v == .string) v.string else "") else ""; - const sc: f64 = if (parsed.object.get("score")) |v| switch (v) { - .float => |x| x, - .integer => |x| @floatFromInt(x), - else => std.math.nan(f64), - } else std.math.nan(f64); - if (!hex16.ok(sha) or std.math.isNan(sc)) { - ctx.root.emit(.{ .type = "error", .message = "score needs prompt_sha (16 hex chars) and a numeric score" }); - continue; - } - const reqStr = struct { - fn s(o: std.json.ObjectMap, k: []const u8) []const u8 { - return if (o.get(k)) |v| (if (v == .string) v.string else "") else ""; - } - }; - // SCORE SCALE CONTRACT (issue #168 Gap 4): the canonical wire - // scale is [0,1]. An explicit "scale" field overrides the - // heuristic (review F9): "percent" always divides by 100 and - // accepts values up to 100 (so a percent-scale 0.5 means - // 0.005, not 0.5); "unit" requires [0,1] as-is. Absent, the - // heuristic applies: [0,1] passes, (1,100] is read as a - // percentage and divides by 100, and anything else is rejected - // here so the backend never has to clamp a 43 into a fake 1.0. - const scale = reqStr.s(parsed.object, "scale"); - // An unrecognized scale must fail loudly, not fall through to - // the heuristic — a typo like "Percent" silently reinterpreting - // 0.7-of-100 as unit-scale 0.7 is a 100x corruption. - if (scale.len > 0 and !std.mem.eql(u8, scale, "percent") and !std.mem.eql(u8, scale, "unit")) { - ctx.root.emit(.{ .type = "error", .message = "unknown scale: use \"unit\" or \"percent\" (or omit it for the [0,1]/(1,100] heuristic)" }); - continue; - } - const normalized: ?f64 = if (std.mem.eql(u8, scale, "percent")) - (if (sc < 0 or sc > 100) null else sc / 100.0) - else if (std.mem.eql(u8, scale, "unit")) - (if (sc < 0 or sc > 1) null else sc) - else - scoring.normalizeOutboundScore(sc); - const sc01 = normalized orelse { - ctx.root.emit(.{ .type = "error", .message = "score out of range: scale=\"unit\" requires [0,1], scale=\"percent\" requires [0,100] (sent as value/100); without scale, [0,1] passes, (1,100] is normalized to /100, and values outside [0,100] are rejected" }); - continue; - }; - const notes = if (parsed.object.get("notes")) |v| (if (v == .string) v.string else "") else ""; - // Optional genome lineage: which prompt this variant was - // mutated from — the children-count input for DGM parent - // selection. - const parent = if (parsed.object.get("parent_sha")) |v| (if (v == .string and hex16.ok(v.string)) v.string else "") else ""; - // Provenance (Step 0): the driver names which judge produced - // the score, the artifact it judged, and the eval-set hash; - // run_id defaults to this session's. All are HMAC-signed so a - // forged trajectory row is detectable. User-controlled fields - // are sanitized (tab/newline/CR → ' ', review F7) BEFORE - // signing so the signed bytes equal the transported bytes — - // an embedded tab would otherwise split the prov transport. - var jid_buf: [64]u8 = undefined; - var art_buf: [64]u8 = undefined; - const judge_id = scoring.sanitizeMetaField(&jid_buf, util.utf8Prefix(reqStr.s(parsed.object, "judge_id"), 64)); - const artifact_sha = scoring.sanitizeMetaField(&art_buf, util.utf8Prefix(reqStr.s(parsed.object, "artifact_sha"), 64)); - // DGM lever: when the score omits eval_set_hash but an --eval suite is - // configured, stamp the suite's stable fingerprint so scores group into a - // promotable (niche × tier × suite) cell. Same --eval cmd → same hash - // across installs → the fleet can rank + promote a champion. - var esh_buf: [16]u8 = undefined; - var eshp_buf: [64]u8 = undefined; - const eval_set_hash = eshblk: { - const provided = scoring.sanitizeMetaField(&eshp_buf, util.utf8Prefix(reqStr.s(parsed.object, "eval_set_hash"), 64)); - if (provided.len > 0) break :eshblk provided; - if (ctx.root.eval_cmd) |c| { - esh_buf = scoring.promptFingerprint(c); - break :eshblk @as([]const u8, &esh_buf); - } - break :eshblk ""; - }; - // run_id is signed too — sanitize it like the other meta - // fields so an embedded newline can't shift the v2 envelope - // (one signature verifying two different field bindings). - var run_buf: [64]u8 = undefined; - const req_run = scoring.sanitizeMetaField(&run_buf, util.utf8Prefix(reqStr.s(parsed.object, "run_id"), 64)); - const run_id: []const u8 = if (req_run.len > 0) req_run else &scoring.g_run_id; - // v2 envelope (issue #168 Gap 2): niche + provider_class are - // signed, so a score for one cell can't be replayed into - // another by mutating transport fields. The niche is truncated - // to 64 (fleetEvent's own cap) and sanitized BEFORE signing so - // the signed bytes match what the backend ingests. - var niche_buf: [64]u8 = undefined; - const req_niche = scoring.sanitizeMetaField(&niche_buf, util.utf8Prefix(reqStr.s(parsed.object, "niche"), 64)); - const pclass = scoring.providerClass(ctx.root.provider.model); - const sig = scoring.signScore(sha, parent, sc01, run_id, judge_id, artifact_sha, eval_set_hash, req_niche, pclass); - const signed = scoring.g_score_key != null; - if (trace.g_traj) |tj| tj.node(.{ - .kind = "score", - .prompt_sha = sha, - .parent_sha = parent, - .score = sc01, - .notes = util.utf8Prefix(notes, 200), - .run_id = run_id, - .judge_id = judge_id, - .artifact_sha = artifact_sha, - .eval_set_hash = eval_set_hash, - .niche = req_niche, - .provider_class = pclass, - .sig = if (signed) @as([]const u8, &sig) else "", - .t = tj.elapsedMs(), - }); - var provbuf: [512]u8 = undefined; - // prov = judge_id, artifact_sha, eval_set_hash + provider_class, niche — - // all five folded into the v2 HMAC — so harness_scores can form - // (niche x provider_class x eval_set_hash) cells the fleet ranks over. - const prov = std.fmt.bufPrint(&provbuf, "{s}\t{s}\t{s}\t{s}\t{s}", .{ judge_id, artifact_sha, eval_set_hash, pclass, req_niche }) catch ""; - if (telemetry.g_telem) |t| t.scoreEvent(sha, parent, sc01, run_id, if (signed) @as([]const u8, &sig) else "", prov); - // fleet:submit (docs §9.B) — a scored, pinned-eval variant entered the fleet grid. - if (eval_set_hash.len > 0) if (telemetry.g_telem) |t| t.fleetEvent("submit", req_niche, sha, "", pclass, eval_set_hash, 0, ""); - ctx.root.emit(.{ .type = "score", .ok = true, .prompt_sha = sha, .signed = signed }); + mainloop_score.handle(ctx.root, parsed.object); continue; } if (std.mem.eql(u8, rtype, "answer")) { @@ -433,6 +321,7 @@ pub fn run(ctx: *Ctx) !void { else try ctx.arena.dupe(u8, text); ctx.root.sys_strict = try std.fmt.allocPrint(ctx.arena, "{s}{s}", .{ ctx.root.sys_normal, prompts.strict_note }); + ctx.root.rebaseContextMeter(); ctx.root.emit(.{ .type = "system_prompt", .ok = true, .append = append, .chars = ctx.root.sys_normal.len }); continue; } @@ -489,15 +378,18 @@ pub fn run(ctx: *Ctx) !void { // model can see (otherwise it only gets the path and resorts to OCR). vision.stageGuiImageAttachment(ctx.root, msg); - // Generate the AI tab-title concurrently (io.async), spawned BEFORE the - // header so the first-turn header can wait for the summary title. Applied - // after the turn on the happy path (below); the defer is the fallback for + // Generate the AI tab-title concurrently, spawned BEFORE the header and + // root turn. `concurrent` guarantees an independent execution slot on + // supported Io backends; async preserves portability as the fallback. + // Applied after the turn on the happy path (below); the defer is the fallback for // an interrupted/stalled/dropped/errored turn (those `continue` before the // apply step) so the generated title is never silently discarded. var title_fut: ?Io.Future(?[]const u8) = null; if (!main_mod.json_mode and ctx.root.ai_title and !ctx.root.ai_title_done) { ctx.root.ai_title_done = true; - title_fut = ctx.io.async(title_mod.titleTask, .{ ctx.gpa, ctx.io, ctx.root.client, ctx.root.provider, base_msg }); + const title_args = .{ ctx.gpa, ctx.io, ctx.root.client, ctx.root.provider, base_msg }; + title_fut = ctx.io.concurrent(title_mod.titleTask, title_args) catch + ctx.io.async(title_mod.titleTask, title_args); } defer if (title_fut) |*f| { applyAiTitle(ctx, f); @@ -571,7 +463,11 @@ pub fn run(ctx: *Ctx) !void { // A failed turn must never kill the session: ApiError is already // reported inside request(); anything else is surfaced here. Either // way we drop back to the prompt (or emit a JSON error/turn event). - const turn_result = providers.runTurnWithFallback(ctx.root, ctx.keys.*, ctx.arena, ctx.out); + const turn_result = providers.runTurnWithFallback(ctx.root, ctx.keys, ctx.arena, ctx.out); + // No successful-path history mutation occurs between the trajectory, + // terminal event, and compaction gate. Reuse one full-history scan for + // all three instead of serializing an increasingly large history 3x. + const post_turn_context_tokens = ctx.root.effectiveContextTokens(); if (trace.g_traj) |tj| { const fp = scoring.promptFingerprint(ctx.root.systemPrompt()); const turn_ms: i64 = @intCast(@max(0, turn_started.untilNow(ctx.io, .awake).toMilliseconds())); @@ -590,7 +486,7 @@ pub fn run(ctx: *Ctx) !void { .task = util.utf8Prefix(base_msg, 160), .tools = turn_tools, .ok = turn_ok, - .context_tokens = ctx.root.last_context_tokens, + .context_tokens = post_turn_context_tokens, }); // Preserve the failure reason in the archive: the turn node only // records ok:false, so an adjacent error record keeps the @@ -640,7 +536,8 @@ pub fn run(ctx: *Ctx) !void { ctx.root.emit(.{ .type = "error", .message = ctx.root.last_api_error orelse "stream stalled — ending turn" }); if (partial.len > 0) { ctx.root.emit(.{ .type = "finalizing" }); - ctx.root.emit(.{ .type = "turn", .text = partial, .context_tokens = ctx.root.last_context_tokens, .cost_usd = pricing.g_cost.snap(ctx.io).usd, .complete = false, .metadata_complete = ctx.root.last_context_tokens > 0 }); + const context_tokens = ctx.root.effectiveContextTokens(); + ctx.root.emit(.{ .type = "turn", .text = partial, .context_tokens = context_tokens, .cost_usd = pricing.g_cost.snap(ctx.io).usd, .complete = false, .metadata_complete = context_tokens > 0 }); } } session.saveSession(ctx.root, ctx.arena, ctx.root.session_name) catch {}; @@ -661,7 +558,8 @@ pub fn run(ctx: *Ctx) !void { ctx.root.emit(.{ .type = "error", .message = ctx.root.last_api_error orelse "stream dropped — ending turn" }); if (partial.len > 0) { ctx.root.emit(.{ .type = "finalizing" }); - ctx.root.emit(.{ .type = "turn", .text = partial, .context_tokens = ctx.root.last_context_tokens, .cost_usd = pricing.g_cost.snap(ctx.io).usd, .complete = false, .metadata_complete = ctx.root.last_context_tokens > 0 }); + const context_tokens = ctx.root.effectiveContextTokens(); + ctx.root.emit(.{ .type = "turn", .text = partial, .context_tokens = context_tokens, .cost_usd = pricing.g_cost.snap(ctx.io).usd, .complete = false, .metadata_complete = context_tokens > 0 }); } } session.saveSession(ctx.root, ctx.arena, ctx.root.session_name) catch {}; @@ -674,13 +572,19 @@ pub fn run(ctx: *Ctx) !void { const partial = std.mem.trim(u8, ctx.root.partial_text.items, " \t\r\n"); if (partial.len > 0) { ctx.root.emit(.{ .type = "finalizing" }); - ctx.root.emit(.{ .type = "turn", .text = partial, .context_tokens = ctx.root.last_context_tokens, .cost_usd = pricing.g_cost.snap(ctx.io).usd, .complete = false, .metadata_complete = ctx.root.last_context_tokens > 0 }); + const context_tokens = ctx.root.effectiveContextTokens(); + ctx.root.emit(.{ .type = "turn", .text = partial, .context_tokens = context_tokens, .cost_usd = pricing.g_cost.snap(ctx.io).usd, .complete = false, .metadata_complete = context_tokens > 0 }); } } // A turn can fail because the context window overflowed; if we're - // over the compaction threshold, compact (or emergency-trim) now - // so the next turn isn't doomed to fail at the same size (#88). - if (ctx.root.last_context_tokens >= ctx.root.provider.compactAt()) ctx.root.compactOrRecover(true); + // over the compaction threshold, compact now so the next turn + // isn't doomed to fail at the same size (#88). Match every other + // automatic compaction path: a failed summary may destructively + // trim only when the best meter says we're genuinely near 95%. + const recovery_meter = ctx.root.effectiveContextTokens(); + if (recovery_meter >= ctx.root.provider.compactAt()) { + ctx.root.compactOrRecover(ctx.root.provider.nearContextLimit(recovery_meter)); + } session.saveSession(ctx.root, ctx.arena, ctx.root.session_name) catch {}; continue; }, @@ -701,15 +605,18 @@ pub fn run(ctx: *Ctx) !void { else final_text; ctx.root.emit(.{ .type = "finalizing" }); - ctx.root.emit(.{ .type = "turn", .text = emitted_text, .context_tokens = ctx.root.last_context_tokens, .cost_usd = pricing.g_cost.snap(ctx.io).usd, .complete = true, .metadata_complete = ctx.root.last_context_tokens > 0 }); + const context_tokens = post_turn_context_tokens; // #124: allocator-level leak telemetry (GRAFF_MEM_DEBUG=1) — arena // capacity per turn separates a session-arena leak from gpa-side - // growth, which OS-level RSS sampling can't tell apart. + // growth, which OS-level RSS sampling can't tell apart. Emit this + // before the terminal turn event so protocol bridges that stop at + // `complete:true` never strand a debug event for the next request. if (main_mod.g_mem_debug) ctx.root.emit(.{ .type = "mem", .session_arena_kb = if (main_mod.g_session_arena) |a| a.queryCapacity() / 1024 else 0, .scratch_arena_kb = if (ctx.root.scratch_arena) |a| a.queryCapacity() / 1024 else 0, }); + ctx.root.emit(.{ .type = "turn", .text = emitted_text, .context_tokens = context_tokens, .cost_usd = pricing.g_cost.snap(ctx.io).usd, .complete = true, .metadata_complete = context_tokens > 0 }); } // Apply the AI summary title + fleet champions that ran in the background @@ -731,10 +638,11 @@ pub fn run(ctx: *Ctx) !void { } } - if (ctx.root.last_context_tokens >= ctx.root.provider.compactAt()) { + const context_tokens = post_turn_context_tokens; + if (context_tokens >= ctx.root.provider.compactAt()) { // Trim on failure only when we're genuinely against the window — at // 80–95% a transient compaction failure can recover next turn. - const near_cap = ctx.root.provider.context > 0 and ctx.root.last_context_tokens * 100 >= ctx.root.provider.context * 95; + const near_cap = ctx.root.provider.nearContextLimit(context_tokens); ctx.root.compactOrRecover(near_cap); } diff --git a/src/mainloop_score.zig b/src/mainloop_score.zig new file mode 100644 index 00000000..ca4399e3 --- /dev/null +++ b/src/mainloop_score.zig @@ -0,0 +1,112 @@ +//! Validation, provenance signing, and event emission for JSON-protocol score +//! requests. Kept separate from the interactive main loop so the protocol's +//! scale and signature contract has one focused implementation. + +const std = @import("std"); + +const agent_mod = @import("agent.zig"); +const scoring = @import("scoring.zig"); +const telemetry = @import("telemetry.zig"); +const trace = @import("trace.zig"); +const util = @import("util.zig"); + +fn isHex16(s: []const u8) bool { + if (s.len != 16) return false; + for (s) |c| if (!std.ascii.isHex(c)) return false; + return true; +} + +fn stringField(obj: std.json.ObjectMap, key: []const u8) []const u8 { + return if (obj.get(key)) |value| (if (value == .string) value.string else "") else ""; +} + +/// Handle one JSON-protocol score request. Validation failures are protocol +/// errors, so they are emitted to the caller rather than returned as Zig errors. +pub fn handle(root: *agent_mod.Agent, obj: std.json.ObjectMap) void { + // Fingerprint fields must be exactly 16 hex chars. A length-only check + // would let an embedded newline shift fields in the signed v2 envelope. + const sha = stringField(obj, "prompt_sha"); + const raw_score: f64 = if (obj.get("score")) |value| switch (value) { + .float => |number| number, + .integer => |number| @floatFromInt(number), + else => std.math.nan(f64), + } else std.math.nan(f64); + if (!isHex16(sha) or std.math.isNan(raw_score)) { + root.emit(.{ .type = "error", .message = "score needs prompt_sha (16 hex chars) and a numeric score" }); + return; + } + + // The canonical wire scale is [0,1]. An explicit percent scale always + // divides by 100; absent a scale, [0,1] passes and (1,100] is normalized. + const scale = stringField(obj, "scale"); + if (scale.len > 0 and !std.mem.eql(u8, scale, "percent") and !std.mem.eql(u8, scale, "unit")) { + root.emit(.{ .type = "error", .message = "unknown scale: use \"unit\" or \"percent\" (or omit it for the [0,1]/(1,100] heuristic)" }); + return; + } + const normalized: ?f64 = if (std.mem.eql(u8, scale, "percent")) + (if (raw_score < 0 or raw_score > 100) null else raw_score / 100.0) + else if (std.mem.eql(u8, scale, "unit")) + (if (raw_score < 0 or raw_score > 1) null else raw_score) + else + scoring.normalizeOutboundScore(raw_score); + const score = normalized orelse { + root.emit(.{ .type = "error", .message = "score out of range: scale=\"unit\" requires [0,1], scale=\"percent\" requires [0,100] (sent as value/100); without scale, [0,1] passes, (1,100] is normalized to /100, and values outside [0,100] are rejected" }); + return; + }; + + const notes = stringField(obj, "notes"); + const parent = if (obj.get("parent_sha")) |value| (if (value == .string and isHex16(value.string)) value.string else "") else ""; + + // Sanitize user-controlled provenance before signing so the signed bytes + // exactly match the transported bytes. + var judge_buf: [64]u8 = undefined; + var artifact_buf: [64]u8 = undefined; + const judge_id = scoring.sanitizeMetaField(&judge_buf, util.utf8Prefix(stringField(obj, "judge_id"), 64)); + const artifact_sha = scoring.sanitizeMetaField(&artifact_buf, util.utf8Prefix(stringField(obj, "artifact_sha"), 64)); + + // Stamp the configured eval suite when the request omits a hash, allowing + // scores to group into a promotable niche/provider/suite cell. + var derived_eval_hash: [16]u8 = undefined; + var eval_hash_buf: [64]u8 = undefined; + const eval_set_hash = eval_hash: { + const provided = scoring.sanitizeMetaField(&eval_hash_buf, util.utf8Prefix(stringField(obj, "eval_set_hash"), 64)); + if (provided.len > 0) break :eval_hash provided; + if (root.eval_cmd) |command| { + derived_eval_hash = scoring.promptFingerprint(command); + break :eval_hash @as([]const u8, &derived_eval_hash); + } + break :eval_hash ""; + }; + + var run_buf: [64]u8 = undefined; + const requested_run = scoring.sanitizeMetaField(&run_buf, util.utf8Prefix(stringField(obj, "run_id"), 64)); + const run_id: []const u8 = if (requested_run.len > 0) requested_run else &scoring.g_run_id; + + var niche_buf: [64]u8 = undefined; + const niche = scoring.sanitizeMetaField(&niche_buf, util.utf8Prefix(stringField(obj, "niche"), 64)); + const provider_class = scoring.providerClass(root.provider.model); + const signature = scoring.signScore(sha, parent, score, run_id, judge_id, artifact_sha, eval_set_hash, niche, provider_class); + const signed = scoring.g_score_key != null; + + if (trace.g_traj) |trajectory| trajectory.node(.{ + .kind = "score", + .prompt_sha = sha, + .parent_sha = parent, + .score = score, + .notes = util.utf8Prefix(notes, 200), + .run_id = run_id, + .judge_id = judge_id, + .artifact_sha = artifact_sha, + .eval_set_hash = eval_set_hash, + .niche = niche, + .provider_class = provider_class, + .sig = if (signed) @as([]const u8, &signature) else "", + .t = trajectory.elapsedMs(), + }); + + var provenance_buf: [512]u8 = undefined; + const provenance = std.fmt.bufPrint(&provenance_buf, "{s}\t{s}\t{s}\t{s}\t{s}", .{ judge_id, artifact_sha, eval_set_hash, provider_class, niche }) catch ""; + if (telemetry.g_telem) |item| item.scoreEvent(sha, parent, score, run_id, if (signed) @as([]const u8, &signature) else "", provenance); + if (eval_set_hash.len > 0) if (telemetry.g_telem) |item| item.fleetEvent("submit", niche, sha, "", provider_class, eval_set_hash, 0, ""); + root.emit(.{ .type = "score", .ok = true, .prompt_sha = sha, .signed = signed }); +} diff --git a/src/models_cache.zig b/src/models_cache.zig index 3d4cfecc..6d9a48fc 100644 --- a/src/models_cache.zig +++ b/src/models_cache.zig @@ -45,6 +45,25 @@ const codex_cache_ttl_ms: i64 = 5 * 60 * 1000; const codex_client_version_floor = "0.144.1"; pub var codex_catalog_source: []const u8 = "baked offline fallback"; +/// Session-owned demand loader. Non-Codex launches keep the baked rows until +/// a picker, switch, resume, or fallback can actually observe Codex models. +/// The loader is invalidated after an in-session Codex login so the new +/// account receives its own catalog before selection. +pub const LazyCodexCatalog = struct { + codex_home: []const u8, + loaded: bool = false, + + pub noinline fn ensure(self: *LazyCodexCatalog, io: Io, gpa: Allocator, arena: Allocator, home: []const u8, token: []const u8, account: []const u8) void { + if (self.loaded) return; + loadCodexCatalog(io, gpa, arena, home, self.codex_home, token, account, false); + self.loaded = true; + } + + pub fn invalidate(self: *LazyCodexCatalog) void { + self.loaded = false; + } +}; + /// models.dev provider ids to trust FIRST when a model id appears under /// several providers — a vendor's own listing beats a reseller's markup /// (models.dev keys resellers like requesty/openrouter/vercel alongside the @@ -573,6 +592,16 @@ test "numField reads int and float JSON numbers; u64Field clamps" { try std.testing.expectEqual(@as(u64, 0), u64Field(v.object, "missing")); } +test "lazy Codex catalog can be invalidated after account changes" { + const previous_source = codex_catalog_source; + defer codex_catalog_source = previous_source; + var catalog: LazyCodexCatalog = .{ .codex_home = "" }; + catalog.ensure(std.testing.io, std.testing.allocator, std.testing.allocator, "", "", ""); + try std.testing.expect(catalog.loaded); + catalog.invalidate(); + try std.testing.expect(!catalog.loaded); +} + test "findModel prefers the canonical vendor over a reseller markup" { var a = std.heap.ArenaAllocator.init(std.testing.allocator); defer a.deinit(); diff --git a/src/oauth.zig b/src/oauth.zig index 116ba232..d1c3fc73 100644 --- a/src/oauth.zig +++ b/src/oauth.zig @@ -11,7 +11,14 @@ const std = @import("std"); const Io = std.Io; const Value = std.json.Value; const Allocator = std.mem.Allocator; -const builtin = @import("builtin"); + +const helpers = @import("oauth_helpers.zig"); +const b64url = helpers.b64url; +const accountFromIdToken = helpers.accountFromIdToken; +const writeCodexAuth = helpers.writeCodexAuth; +const codex_login_page_html = helpers.codex_login_page_html; +const queryParam = helpers.queryParam; +const openBrowser = helpers.openBrowser; const ansi = @import("ansi.zig"); const style = &ansi.style; @@ -72,12 +79,6 @@ const codex_redirect = "http://localhost:1455/auth/callback"; const codex_redirect_enc = "http%3A%2F%2Flocalhost%3A1455%2Fauth%2Fcallback"; const oauth_scope_enc = "openid%20profile%20email%20offline_access"; -fn b64url(arena: Allocator, bytes: []const u8) []const u8 { - const enc = std.base64.url_safe_no_pad.Encoder; - const buf = arena.alloc(u8, enc.calcSize(bytes.len)) catch return ""; - return enc.encode(buf, bytes); -} - /// POST a form body to the OAuth token endpoint; return the parsed JSON object. fn oauthTokenPost(io: Io, gpa: Allocator, arena: Allocator, body: []const u8) !std.json.ObjectMap { var client: std.http.Client = .{ .allocator = gpa, .io = io }; @@ -95,77 +96,6 @@ fn oauthTokenPost(io: Io, gpa: Allocator, arena: Allocator, body: []const u8) !s return v.object; } -/// Pull chatgpt_account_id out of an id_token's claims (base64url middle segment). -fn accountFromIdToken(arena: Allocator, id_token: []const u8) []const u8 { - var it = std.mem.splitScalar(u8, id_token, '.'); - _ = it.next(); - const payload = it.next() orelse return ""; - const dec = std.base64.url_safe_no_pad.Decoder; - const n = dec.calcSizeForSlice(payload) catch return ""; - const buf = arena.alloc(u8, n) catch return ""; - dec.decode(buf, payload) catch return ""; - const v = std.json.parseFromSliceLeaky(Value, arena, buf, .{ .allocate = .alloc_always }) catch return ""; - if (v != .object) return ""; - const a = v.object.get("https://api.openai.com/auth") orelse return ""; - if (a != .object) return ""; - const acct = a.object.get("chatgpt_account_id") orelse return ""; - return if (acct == .string) acct.string else ""; -} - -fn writeCodexAuth(io: Io, arena: Allocator, home: []const u8, id_token: []const u8, access: []const u8, refresh: []const u8, account: []const u8) !void { - const path = try std.fmt.allocPrint(arena, "{s}/.codex/auth.json", .{home}); - var toks: std.json.ObjectMap = .empty; - try toks.put(arena, "id_token", .{ .string = id_token }); - try toks.put(arena, "access_token", .{ .string = access }); - try toks.put(arena, "refresh_token", .{ .string = refresh }); - try toks.put(arena, "account_id", .{ .string = account }); - var obj: std.json.ObjectMap = .empty; - try obj.put(arena, "auth_mode", .{ .string = "chatgpt" }); - try obj.put(arena, "tokens", .{ .object = toks }); - try obj.put(arena, "last_refresh", .{ .string = "harness-login" }); - var aw: Io.Writer.Allocating = .init(arena); - var s: std.json.Stringify = .{ .writer = &aw.writer }; - try s.write(Value{ .object = obj }); - const f = try Io.Dir.cwd().createFile(io, path, .{}); - defer f.close(io); - var wbuf: [4096]u8 = undefined; - var fw = f.writer(io, &wbuf); - try fw.interface.writeAll(aw.writer.buffered()); - try fw.interface.flush(); -} - -/// The page the Codex OAuth callback tab lands on after a successful login. -/// A branded, dark-mode-aware confirmation card; the local server writes this -/// back, then reads the ?code out of the same request for the token exchange. -const codex_login_page_html = - \\ - \\ - \\ - \\graff — logged in - \\ - \\
- \\
graff
- \\

You're all set

- \\

Codex is connected. You can close this tab and return to graff.

- \\
-; /// `harness login [--refresh]`: the ChatGPT/Codex OAuth flow. Fresh login runs /// PKCE (open browser → localhost:1455 callback → code→token exchange); refresh /// uses the stored refresh_token. Either way writes ~/.codex/auth.json. @@ -267,27 +197,6 @@ pub fn codexLogin(io: Io, gpa: Allocator, arena: Allocator, home: []const u8, re try out.flush(); } -/// Extract a query-string parameter from an HTTP request line ("GET /p?k=v…"). -fn queryParam(req_line: []const u8, key: []const u8) ?[]const u8 { - const q = std.mem.indexOfScalar(u8, req_line, '?') orelse return null; - const sp = std.mem.indexOfScalarPos(u8, req_line, q, ' ') orelse req_line.len; - var it = std.mem.tokenizeScalar(u8, req_line[q + 1 .. sp], '&'); - while (it.next()) |pair| { - const eq = std.mem.indexOfScalar(u8, pair, '=') orelse continue; - if (std.mem.eql(u8, pair[0..eq], key)) return pair[eq + 1 ..]; - } - return null; -} - -fn openBrowser(io: Io, url: []const u8) void { - const argv: []const []const u8 = if (builtin.os.tag == .macos) - &.{ "open", url } - else - &.{ "xdg-open", url }; - var child = std.process.spawn(io, .{ .argv = argv, .stdin = .ignore, .stdout = .ignore, .stderr = .ignore }) catch return; - _ = child.wait(io) catch {}; -} - // Kimi Code OAuth — device-code flow, same client + endpoints the Kimi CLI // uses (auth.kimi.com). `graff login kimi` runs the flow and writes the token // to ~/.kimi/credentials/graff-oauth.json; loadKimiOAuth reads it at startup diff --git a/src/oauth_helpers.zig b/src/oauth_helpers.zig new file mode 100644 index 00000000..9dad99c8 --- /dev/null +++ b/src/oauth_helpers.zig @@ -0,0 +1,107 @@ +//! Codex callback parsing and common browser helpers used by OAuth login flows. + +const std = @import("std"); +const builtin = @import("builtin"); + +const Io = std.Io; +const Value = std.json.Value; +const Allocator = std.mem.Allocator; + +pub fn b64url(arena: Allocator, bytes: []const u8) []const u8 { + const enc = std.base64.url_safe_no_pad.Encoder; + const buf = arena.alloc(u8, enc.calcSize(bytes.len)) catch return ""; + return enc.encode(buf, bytes); +} + +/// Pull chatgpt_account_id out of an id_token's claims (base64url middle segment). +pub fn accountFromIdToken(arena: Allocator, id_token: []const u8) []const u8 { + var it = std.mem.splitScalar(u8, id_token, '.'); + _ = it.next(); + const payload = it.next() orelse return ""; + const dec = std.base64.url_safe_no_pad.Decoder; + const n = dec.calcSizeForSlice(payload) catch return ""; + const buf = arena.alloc(u8, n) catch return ""; + dec.decode(buf, payload) catch return ""; + const v = std.json.parseFromSliceLeaky(Value, arena, buf, .{ .allocate = .alloc_always }) catch return ""; + if (v != .object) return ""; + const auth = v.object.get("https://api.openai.com/auth") orelse return ""; + if (auth != .object) return ""; + const account = auth.object.get("chatgpt_account_id") orelse return ""; + return if (account == .string) account.string else ""; +} + +pub fn writeCodexAuth(io: Io, arena: Allocator, home: []const u8, id_token: []const u8, access: []const u8, refresh: []const u8, account: []const u8) !void { + const path = try std.fmt.allocPrint(arena, "{s}/.codex/auth.json", .{home}); + var tokens: std.json.ObjectMap = .empty; + try tokens.put(arena, "id_token", .{ .string = id_token }); + try tokens.put(arena, "access_token", .{ .string = access }); + try tokens.put(arena, "refresh_token", .{ .string = refresh }); + try tokens.put(arena, "account_id", .{ .string = account }); + var object: std.json.ObjectMap = .empty; + try object.put(arena, "auth_mode", .{ .string = "chatgpt" }); + try object.put(arena, "tokens", .{ .object = tokens }); + try object.put(arena, "last_refresh", .{ .string = "harness-login" }); + var aw: Io.Writer.Allocating = .init(arena); + var stringify: std.json.Stringify = .{ .writer = &aw.writer }; + try stringify.write(Value{ .object = object }); + const file = try Io.Dir.cwd().createFile(io, path, .{}); + defer file.close(io); + var write_buffer: [4096]u8 = undefined; + var writer = file.writer(io, &write_buffer); + try writer.interface.writeAll(aw.writer.buffered()); + try writer.interface.flush(); +} + +/// The page the Codex OAuth callback tab lands on after a successful login. +/// A branded, dark-mode-aware confirmation card; the local server writes this +/// back, then reads the ?code out of the same request for the token exchange. +pub const codex_login_page_html = + \\ + \\ + \\ + \\graff — logged in + \\ + \\
+ \\
graff
+ \\

You're all set

+ \\

Codex is connected. You can close this tab and return to graff.

+ \\
+; + +/// Extract a query-string parameter from an HTTP request line ("GET /p?k=v…"). +pub fn queryParam(req_line: []const u8, key: []const u8) ?[]const u8 { + const query_start = std.mem.indexOfScalar(u8, req_line, '?') orelse return null; + const request_end = std.mem.indexOfScalarPos(u8, req_line, query_start, ' ') orelse req_line.len; + var pairs = std.mem.tokenizeScalar(u8, req_line[query_start + 1 .. request_end], '&'); + while (pairs.next()) |pair| { + const equals = std.mem.indexOfScalar(u8, pair, '=') orelse continue; + if (std.mem.eql(u8, pair[0..equals], key)) return pair[equals + 1 ..]; + } + return null; +} + +pub fn openBrowser(io: Io, url: []const u8) void { + const argv: []const []const u8 = if (builtin.os.tag == .macos) + &.{ "open", url } + else + &.{ "xdg-open", url }; + var child = std.process.spawn(io, .{ .argv = argv, .stdin = .ignore, .stdout = .ignore, .stderr = .ignore }) catch return; + _ = child.wait(io) catch {}; +} diff --git a/src/picker_auth.zig b/src/picker_auth.zig new file mode 100644 index 00000000..d89de330 --- /dev/null +++ b/src/picker_auth.zig @@ -0,0 +1,222 @@ +//! Provider authentication flow used by the interactive model picker. +//! The picker UI itself stays in pickers.zig; it is injected here to avoid +//! making this helper depend back on the module that re-exports it. + +const std = @import("std"); +const Io = std.Io; +const Allocator = std.mem.Allocator; + +const ansi = @import("ansi.zig"); +const style = &ansi.style; +const term = @import("term.zig"); +const tty = term.tty; +const oauth = @import("oauth.zig"); +const providers = @import("providers.zig"); +const switchProvider = providers.switchProvider; +const agent_mod = @import("agent.zig"); +const provider_mod = @import("provider.zig"); +const keys_cli = @import("keys_cli.zig"); +const command_catalog = @import("command_catalog.zig"); +const pricing = @import("pricing.zig"); + +const Agent = agent_mod.Agent; +const Keys = provider_mod.Keys; +const provider_specs = provider_mod.provider_specs; +const storeKey = keys_cli.storeKey; +const PickItem = command_catalog.Item; + +pub const PickerFn = *const fn ( + root: *Agent, + arena: Allocator, + out: *Io.Writer, + title: []const u8, + items: []const PickItem, +) ?usize; + +/// After an in-session `/login` writes its credential file, pull the fresh key +/// (and the Codex account id) into the live Keys so the current conversation +/// uses it without a restart — the in-session twin of the startup loaders. +pub fn reloadLoginKey(root: *Agent, keys: *Keys, arena: Allocator, provider_id: []const u8) void { + const home = root.home; + for (provider_specs, &keys.values, &keys.sources) |spec, *value, *source| { + if (!std.mem.eql(u8, spec.id, provider_id)) continue; + if (std.mem.eql(u8, provider_id, "codegraff")) { + if (oauth.loadCodegraffKey(root.io, arena, home)) |k| { + value.* = k; + source.* = .login; + } + } else if (std.mem.eql(u8, provider_id, "kimi")) { + if (oauth.loadKimiOAuth(root.io, root.gpa, arena, home, false)) |k| { + value.* = k; + source.* = .login; + } + } else if (std.mem.eql(u8, provider_id, "xai")) { + if (oauth.loadXaiOAuth(root.io, root.gpa, arena, home, false)) |k| { + value.* = k; + source.* = .login; + } + } else if (std.mem.eql(u8, provider_id, "codex")) { + if (oauth.loadCodexAuth(root.io, arena, home)) |auth| { + value.* = auth.token; + source.* = .login; + keys.codex_account = auth.account; + } + } + } + if (std.mem.eql(u8, provider_id, "codex") and keys.get("codex") != null) + root.reloadModelCatalog(keys.*); +} + +/// Read an API key without terminal echo or history persistence. The ordinary +/// REPL line editor masks `/key provider secret`; this covers the model +/// picker's separate "paste an API key" flow too. +fn readSecret(root: *Agent, arena: Allocator, out: *Io.Writer) ?[]const u8 { + const in = root.in orelse return null; + const raw_state = tty.enterRaw(true) orelse return null; + defer tty.restore(raw_state); + var secret: std.ArrayList(u8) = .empty; + while (true) { + const byte = in.takeByte() catch return null; + switch (byte) { + '\r', '\n' => { + out.writeByte('\n') catch {}; + out.flush() catch {}; + return secret.toOwnedSlice(arena) catch secret.items; + }, + 0x03, 0x07 => { + out.writeByte('\n') catch {}; + out.flush() catch {}; + return null; + }, + 0x7f, 0x08 => if (secret.items.len > 0) { + _ = secret.pop(); + out.writeAll("\x08 \x08") catch {}; + out.flush() catch {}; + }, + else => if (byte >= 0x20) { + secret.append(arena, byte) catch return null; + out.writeByte('*') catch {}; + out.flush() catch {}; + }, + } + } +} + +/// Better UX when /model targets a provider with no key: offer OAuth login or +/// key entry and then switch to the requested provider/model. `pick` is the +/// generic picker owned by pickers.zig. +pub fn offerProviderAuth( + root: *Agent, + keys: *Keys, + arena: Allocator, + out: *Io.Writer, + pid: []const u8, + model: []const u8, + default_selection: bool, + use_color: bool, + pick: PickerFn, +) !void { + var spec_idx: ?usize = null; + for (provider_specs, 0..) |spec, i| if (std.mem.eql(u8, spec.id, pid)) { + spec_idx = i; + }; + const si = spec_idx orelse { + try out.print("unknown provider '{s}' — see /model for the list\n", .{pid}); + try out.flush(); + return; + }; + const can_login = std.mem.eql(u8, pid, "codegraff") or std.mem.eql(u8, pid, "codex") or std.mem.eql(u8, pid, "kimi") or std.mem.eql(u8, pid, "xai"); + + // Non-interactive (one-shot / no TTY): no picker — print the hint and bail. + if (!use_color or root.in == null) { + if (can_login) + try out.print("no key for {s} — /login {s} (OAuth) or /key {s} \n", .{ pid, pid, pid }) + else + try out.print("no key for {s} — /key {s} (or set {s})\n", .{ pid, pid, provider_specs[si].env_key }); + try out.flush(); + return; + } + + // Choice menu — login row only when the provider actually has an OAuth flow. + var items: [3]PickItem = undefined; + var n: usize = 0; + if (can_login) { + items[n] = .{ .name = "log in (OAuth)", .desc = "device/browser sign-in — no key to paste" }; + n += 1; + } + items[n] = .{ .name = "paste an API key", .desc = "enter a key now (used live + saved)" }; + n += 1; + items[n] = .{ .name = "keep current model", .desc = "cancel — stay on the current model" }; + n += 1; + + const title = std.fmt.allocPrint(arena, "No key for {s} \xe2\x80\xba", .{pid}) catch "No key \xe2\x80\xba"; + const choice = pick(root, arena, out, title, items[0..n]) orelse { + try out.print("kept {s}{s}{s}\n", .{ style.cyan, root.provider.model, style.reset }); + try out.flush(); + return; + }; + const picked = items[choice].name; + + if (std.mem.eql(u8, picked, "keep current model")) { + try out.print("kept {s}{s}{s}\n", .{ style.cyan, root.provider.model, style.reset }); + try out.flush(); + return; + } + + if (std.mem.eql(u8, picked, "log in (OAuth)")) { + const home = root.home; + try out.flush(); // hand stdout to the login flow's own writer + if (std.mem.eql(u8, pid, "codegraff")) { + oauth.codegraffLogin(root.io, root.gpa, arena, home) catch |err| { + try out.print("\xe2\x9c\x97 codegraff login failed: {t}\n", .{err}); + try out.flush(); + return; + }; + } else if (std.mem.eql(u8, pid, "codex")) { + oauth.codexLogin(root.io, root.gpa, arena, home, false) catch |err| { + try out.print("\xe2\x9c\x97 codex login failed: {t}\n", .{err}); + try out.flush(); + return; + }; + } else if (std.mem.eql(u8, pid, "kimi")) { + oauth.kimiLogin(root.io, root.gpa, arena, home) catch |err| { + try out.print("\xe2\x9c\x97 kimi login failed: {t}\n", .{err}); + try out.flush(); + return; + }; + } else if (std.mem.eql(u8, pid, "xai")) { + oauth.xaiLogin(root.io, root.gpa, arena, home) catch |err| { + try out.print("\xe2\x9c\x97 xai login failed: {t}\n", .{err}); + try out.flush(); + return; + }; + } + reloadLoginKey(root, keys, arena, pid); + } else { + try out.print("paste your {s} API key, then Enter (input is hidden; blank cancels): ", .{pid}); + try out.flush(); + const key = readSecret(root, arena, out) orelse ""; + if (key.len == 0) { + try out.print("cancelled — kept {s}{s}{s}\n", .{ style.cyan, root.provider.model, style.reset }); + try out.flush(); + return; + } + const dup = arena.dupe(u8, key) catch key; + keys.values[si] = dup; + const saved = storeKey(root.io, root.gpa, arena, root.home, pid, dup); + keys.sources[si] = if (saved) .stored else .session; + try out.print("\xe2\x9c\x93 {s} key set (live{s})\n", .{ pid, if (saved) " + Keychain" else "" }); + } + + // Auth done — switch now if the key/login took, else keep the current model. + const selected_model = if (default_selection and std.mem.eql(u8, pid, "codex")) + pricing.providerDefaultModel(pid, provider_specs[si].default_model) + else + model; + const provider = keys.providerById(pid, selected_model) catch { + try out.print("still no usable key for {s} — kept {s}{s}{s}\n", .{ pid, style.cyan, root.provider.model, style.reset }); + try out.flush(); + return; + }; + try switchProvider(root, arena, provider, out); +} diff --git a/src/pickers.zig b/src/pickers.zig index 940dbb7e..f15cd0e4 100644 --- a/src/pickers.zig +++ b/src/pickers.zig @@ -1,12 +1,9 @@ //! Interactive fuzzy pickers (the /model picker + the generic listPicker //! used by /resume, /login, the bare "/" command menu, etc.), the fuzzy //! match/score/sort primitives behind them, ultracode steering + its on/off -//! toggle picker, the slash-command menu, and the "no key for this -//! provider" login/paste-a-key flow. Split out of main.zig (600-line goal). -//! Back-imports switchProvider from providers.zig (offerProviderAuth's -//! final step) and main (as main_mod, since several params are named -//! `root`) for Agent, Keys, provider_specs, storeKey, and the live -//! use_color toggle. +//! toggle picker, and the slash-command menu. The model picker's provider +//! authentication flow lives in picker_auth.zig. Split out of main.zig +//! (600-line goal). const std = @import("std"); const Io = std.Io; @@ -20,20 +17,13 @@ const tty = term.tty; const pricing = @import("pricing.zig"); -const oauth = @import("oauth.zig"); - -const providers = @import("providers.zig"); -const switchProvider = providers.switchProvider; - const main_mod = @import("main.zig"); const agent_mod = @import("agent.zig"); const provider_mod = @import("provider.zig"); -const keys_cli = @import("keys_cli.zig"); const Agent = agent_mod.Agent; const Keys = provider_mod.Keys; -const provider_specs = provider_mod.provider_specs; -const storeKey = keys_cli.storeKey; const command_catalog = @import("command_catalog.zig"); +const picker_auth = @import("picker_auth.zig"); /// Case-insensitive subsequence match (fzf-style): every char of `needle` /// appears in `hay` in order, gaps allowed — so "gpt5.5" matches "gpt-5.5". @@ -495,177 +485,13 @@ pub fn pickUltracodeMode(root: *Agent, arena: Allocator, out: *Io.Writer) ?bool /// one-line description, picked via listPicker. Returns the command to run. pub const command_menu = command_catalog.commands; -/// After an in-session `/login` writes its credential file, pull the fresh key -/// (and the Codex account id) into the live Keys so the current conversation -/// uses it without a restart — the in-session twin of the startup loaders. -pub fn reloadLoginKey(root: *Agent, keys: *Keys, arena: Allocator, provider_id: []const u8) void { - const home = root.home; - for (provider_specs, &keys.values, &keys.sources) |spec, *value, *source| { - if (!std.mem.eql(u8, spec.id, provider_id)) continue; - if (std.mem.eql(u8, provider_id, "codegraff")) { - if (oauth.loadCodegraffKey(root.io, arena, home)) |k| { - value.* = k; - source.* = .login; - } - } else if (std.mem.eql(u8, provider_id, "kimi")) { - if (oauth.loadKimiOAuth(root.io, root.gpa, arena, home, false)) |k| { - value.* = k; - source.* = .login; - } - } else if (std.mem.eql(u8, provider_id, "xai")) { - if (oauth.loadXaiOAuth(root.io, root.gpa, arena, home, false)) |k| { - value.* = k; - source.* = .login; - } - } else if (std.mem.eql(u8, provider_id, "codex")) { - if (oauth.loadCodexAuth(root.io, arena, home)) |auth| { - value.* = auth.token; - source.* = .login; - keys.codex_account = auth.account; - } - } - } -} - -/// Read an API key without terminal echo or history persistence. The ordinary -/// REPL line editor masks `/key provider secret`; this covers the model -/// picker's separate "paste an API key" flow too. -fn readSecret(root: *Agent, arena: Allocator, out: *Io.Writer) ?[]const u8 { - const in = root.in orelse return null; - const raw_state = tty.enterRaw(true) orelse return null; - defer tty.restore(raw_state); - var secret: std.ArrayList(u8) = .empty; - while (true) { - const byte = in.takeByte() catch return null; - switch (byte) { - '\r', '\n' => { - out.writeByte('\n') catch {}; - out.flush() catch {}; - return secret.toOwnedSlice(arena) catch secret.items; - }, - 0x03, 0x07 => { - out.writeByte('\n') catch {}; - out.flush() catch {}; - return null; - }, - 0x7f, 0x08 => if (secret.items.len > 0) { - _ = secret.pop(); - out.writeAll("\x08 \x08") catch {}; - out.flush() catch {}; - }, - else => if (byte >= 0x20) { - secret.append(arena, byte) catch return null; - out.writeByte('*') catch {}; - out.flush() catch {}; - }, - } - } -} - -/// Better UX when /model targets a provider with no key: instead of a flat -/// "no key" dead-end, offer to log in (OAuth, for providers that have a flow) -/// or paste an API key, then switch to pid/model. Esc/blank/"keep" stays on the -/// current model. Non-TTY just prints the actionable one-liner. Best-effort. -pub fn offerProviderAuth(root: *Agent, keys: *Keys, arena: Allocator, out: *Io.Writer, pid: []const u8, model: []const u8) !void { - var spec_idx: ?usize = null; - for (provider_specs, 0..) |spec, i| if (std.mem.eql(u8, spec.id, pid)) { - spec_idx = i; - }; - const si = spec_idx orelse { - try out.print("unknown provider '{s}' — see /model for the list\n", .{pid}); - try out.flush(); - return; - }; - const can_login = std.mem.eql(u8, pid, "codegraff") or std.mem.eql(u8, pid, "codex") or std.mem.eql(u8, pid, "kimi") or std.mem.eql(u8, pid, "xai"); - - // Non-interactive (one-shot / no TTY): no picker — print the hint and bail. - if (!main_mod.use_color or root.in == null) { - if (can_login) - try out.print("no key for {s} — /login {s} (OAuth) or /key {s} \n", .{ pid, pid, pid }) - else - try out.print("no key for {s} — /key {s} (or set {s})\n", .{ pid, pid, provider_specs[si].env_key }); - try out.flush(); - return; - } - - // Choice menu — login row only when the provider actually has an OAuth flow. - var items: [3]PickItem = undefined; - var n: usize = 0; - if (can_login) { - items[n] = .{ .name = "log in (OAuth)", .desc = "device/browser sign-in — no key to paste" }; - n += 1; - } - items[n] = .{ .name = "paste an API key", .desc = "enter a key now (used live + saved)" }; - n += 1; - items[n] = .{ .name = "keep current model", .desc = "cancel — stay on the current model" }; - n += 1; - - const title = std.fmt.allocPrint(arena, "No key for {s} \xe2\x80\xba", .{pid}) catch "No key \xe2\x80\xba"; - const choice = listPicker(root, arena, out, title, items[0..n]) orelse { - try out.print("kept {s}{s}{s}\n", .{ style.cyan, root.provider.model, style.reset }); - try out.flush(); - return; - }; - const picked = items[choice].name; - - if (std.mem.eql(u8, picked, "keep current model")) { - try out.print("kept {s}{s}{s}\n", .{ style.cyan, root.provider.model, style.reset }); - try out.flush(); - return; - } - - if (std.mem.eql(u8, picked, "log in (OAuth)")) { - const home = root.home; - try out.flush(); // hand stdout to the login flow's own writer - if (std.mem.eql(u8, pid, "codegraff")) { - oauth.codegraffLogin(root.io, root.gpa, arena, home) catch |err| { - try out.print("\xe2\x9c\x97 codegraff login failed: {t}\n", .{err}); - try out.flush(); - return; - }; - } else if (std.mem.eql(u8, pid, "codex")) { - oauth.codexLogin(root.io, root.gpa, arena, home, false) catch |err| { - try out.print("\xe2\x9c\x97 codex login failed: {t}\n", .{err}); - try out.flush(); - return; - }; - } else if (std.mem.eql(u8, pid, "kimi")) { - oauth.kimiLogin(root.io, root.gpa, arena, home) catch |err| { - try out.print("\xe2\x9c\x97 kimi login failed: {t}\n", .{err}); - try out.flush(); - return; - }; - } else if (std.mem.eql(u8, pid, "xai")) { - oauth.xaiLogin(root.io, root.gpa, arena, home) catch |err| { - try out.print("\xe2\x9c\x97 xai login failed: {t}\n", .{err}); - try out.flush(); - return; - }; - } - reloadLoginKey(root, keys, arena, pid); - } else { - try out.print("paste your {s} API key, then Enter (input is hidden; blank cancels): ", .{pid}); - try out.flush(); - const key = readSecret(root, arena, out) orelse ""; - if (key.len == 0) { - try out.print("cancelled — kept {s}{s}{s}\n", .{ style.cyan, root.provider.model, style.reset }); - try out.flush(); - return; - } - const dup = arena.dupe(u8, key) catch key; - keys.values[si] = dup; - const saved = storeKey(root.io, root.gpa, arena, root.home, pid, dup); - keys.sources[si] = if (saved) .stored else .session; - try out.print("\xe2\x9c\x93 {s} key set (live{s})\n", .{ pid, if (saved) " + Keychain" else "" }); - } +pub const reloadLoginKey = picker_auth.reloadLoginKey; - // Auth done — switch now if the key/login took, else keep the current model. - const provider = keys.providerById(pid, model) catch { - try out.print("still no usable key for {s} — kept {s}{s}{s}\n", .{ pid, style.cyan, root.provider.model, style.reset }); - try out.flush(); - return; - }; - try switchProvider(root, arena, provider, out); +/// When a model's provider has no key, offer OAuth login or key entry and then +/// switch to that model. The helper owns authentication while this module owns +/// the picker callback and the live color/TTY mode. +pub fn offerProviderAuth(root: *Agent, keys: *Keys, arena: Allocator, out: *Io.Writer, pid: []const u8, model: []const u8, default_selection: bool) !void { + try picker_auth.offerProviderAuth(root, keys, arena, out, pid, model, default_selection, main_mod.use_color, listPicker); } // Tests moved from main.zig alongside the functions they cover (#123 split). diff --git a/src/pricing.zig b/src/pricing.zig index ee34b16b..282d47d6 100644 --- a/src/pricing.zig +++ b/src/pricing.zig @@ -95,13 +95,13 @@ pub const CostTally = struct { pub fn add(self: *CostTally, io: Io, provider_id: []const u8, model: []const u8, uncached_in: i64, cache_in: i64, out: i64) void { self.mutex.lockUncancelable(io); defer self.mutex.unlock(io); - self.api_calls += 1; - self.in_tokens += @intCast(@max(uncached_in, 0)); - self.cache_tokens += @intCast(@max(cache_in, 0)); - self.out_tokens += @intCast(@max(out, 0)); + self.api_calls +|= 1; + self.in_tokens +|= @intCast(@max(uncached_in, 0)); + self.cache_tokens +|= @intCast(@max(cache_in, 0)); + self.out_tokens +|= @intCast(@max(out, 0)); switch (billingFor(provider_id, model)) { - .sub => self.sub_calls += 1, - .unpriced => self.unpriced_calls += 1, + .sub => self.sub_calls +|= 1, + .unpriced => self.unpriced_calls +|= 1, .priced => self.usd += usdFor(priceFor(model).?, uncached_in, cache_in, out), } } @@ -118,7 +118,7 @@ pub const CostTally = struct { /// One-line summary shared by /cost and the one-shot stderr report. pub fn render(c: CostTally, w: *Io.Writer) !void { try w.print("{d} api call(s) · {d} in ({d} cached) + {d} out tokens · ${d:.4}", .{ - c.api_calls, c.in_tokens + c.cache_tokens, c.cache_tokens, c.out_tokens, c.usd, + c.api_calls, c.in_tokens +| c.cache_tokens, c.cache_tokens, c.out_tokens, c.usd, }); if (c.sub_calls > 0) try w.print(" · {d} subscription call(s), flat-rate (not in $)", .{c.sub_calls}); if (c.unpriced_calls > 0) try w.print(" · {d} call(s) on unpriced models", .{c.unpriced_calls}); @@ -450,6 +450,22 @@ test "usdFor: per-million math and negative clamping" { try std.testing.expectApproxEqAbs(@as(f64, 0.0), usdFor(p, -42, -1, 0), 1e-9); // clamped } +test "CostTally token and call counters saturate" { + var tally: CostTally = .{ + .in_tokens = std.math.maxInt(u64) - 1, + .cache_tokens = std.math.maxInt(u64), + .out_tokens = std.math.maxInt(u64) - 2, + .api_calls = std.math.maxInt(u64), + .sub_calls = std.math.maxInt(u64), + }; + tally.add(std.testing.io, "codex", "gpt-5.5", 10, 10, 10); + try std.testing.expectEqual(std.math.maxInt(u64), tally.in_tokens); + try std.testing.expectEqual(std.math.maxInt(u64), tally.cache_tokens); + try std.testing.expectEqual(std.math.maxInt(u64), tally.out_tokens); + try std.testing.expectEqual(std.math.maxInt(u64), tally.api_calls); + try std.testing.expectEqual(std.math.maxInt(u64), tally.sub_calls); +} + test "modelInTable: known models present, unknown absent" { try std.testing.expect(modelInTable("gpt-5.5")); try std.testing.expect(modelInTable("claude-opus-4-8")); diff --git a/src/provider.zig b/src/provider.zig index ac865363..906c2ccc 100644 --- a/src/provider.zig +++ b/src/provider.zig @@ -109,6 +109,15 @@ pub const Provider = struct { return p.context / 100 * pct; } + /// Whether a failed compaction may safely fall back to destructive trimming. + /// Keep this stricter than compactAt(): at 80-95% a transient summary failure + /// should leave history intact and retry later. Subtraction avoids overflow for + /// provider-controlled token counts near u64.max. + pub fn nearContextLimit(p: Provider, tokens: u64) bool { + if (p.context == 0) return false; + return tokens >= p.context - (p.context / 20); // 95% + } + /// #201: absolute ceiling for a single tool output regardless of window size, /// so a huge-context model still bounds one pathological result. const abs_output_cap_bytes: usize = 256 * 1024; @@ -243,6 +252,16 @@ test "Provider.compactAt: auto-compacts at 80% of the context window (long-horiz try std.testing.expectEqual(@as(u64, 0), zero.compactAt()); } +test "Provider.nearContextLimit: destructive recovery starts at 95% without overflow math" { + const p: Provider = .{ .id = "x", .kind = .openai, .auth = .bearer, .url = "", .api_key = "", .model = "x", .context = 100_000 }; + try std.testing.expect(!p.nearContextLimit(94_999)); + try std.testing.expect(p.nearContextLimit(95_000)); + try std.testing.expect(p.nearContextLimit(std.math.maxInt(u64))); + var unknown = p; + unknown.context = 0; + try std.testing.expect(!unknown.nearContextLimit(std.math.maxInt(u64))); +} + test "Keys.providerFor: known model, claude/gateway fallbacks, missing key" { const all = Keys{ .values = [_]?[]const u8{"k"} ** provider_specs.len }; const none = Keys{ .values = [_]?[]const u8{null} ** provider_specs.len }; diff --git a/src/providers.zig b/src/providers.zig index 5bab6f6b..fe842e73 100644 --- a/src/providers.zig +++ b/src/providers.zig @@ -69,8 +69,13 @@ fn translateHistory(arena: Allocator, msgs: *std.json.Array, to_kind: Provider.K msgs.* = out; } -fn applyProviderInner(root: *Agent, arena: Allocator, p: Provider, persist: bool) []const u8 { +fn applyProviderInner(root: *Agent, arena: Allocator, p: Provider, persist: bool) ![]const u8 { const same_format = root.provider.kind == p.kind; + const same_selection = same_format and + root.provider.context == p.context and + std.mem.eql(u8, root.provider.id, p.id) and + std.mem.eql(u8, root.provider.model, p.model) and + std.mem.eql(u8, root.provider.url, p.url); var note: []const u8 = "context kept"; if (!same_format) { if (root.keep_context) { @@ -79,19 +84,27 @@ fn applyProviderInner(root: *Agent, arena: Allocator, p: Provider, persist: bool } else { root.messages = std.json.Array.init(arena); root.last_context_tokens = 0; + root.context_local_tokens = 0; note = "history cleared — /keepcontext on to carry it across formats"; } } - root.cap_new = false; // per-provider token-cap quirk; relearn on rejection - root.effort_rejected = false; // new model may accept reasoning_effort; relearn - root.ws_off = false; // a previous Codex WS fallback must not leak across switches root.provider = p; - // #204: a provider switch changes the window; don't carry the previous model's - // absolute token count against it. Re-estimate from the (kept or translated) - // history so the meter + compaction gate stay consistent until the next response - // returns real usage. (The cross-format history-clear above already set 0; - // fullInputEstimateTokens over an empty history is 0.) - root.last_context_tokens = root.fullInputEstimateTokens(); + // Keep the context estimate and first request exact without eagerly + // materializing formats this session has never used. + try root.ensureRootTools(p.kind); + // #204: an actual provider/model switch changes the window and tokenizer, so + // re-estimate until the next response returns real usage. A no-op set_model is + // common in the GUI prompt-settings path; preserve its authoritative server + // meter instead of replacing it with the structurally-low local estimate. + if (!same_selection) { + root.cap_new = false; // per-provider token-cap quirk; relearn on rejection + root.effort_rejected = false; // new model may accept reasoning_effort; relearn + root.ws_off = false; // a previous Codex WS fallback must not leak across switches + root.compact_transport_failures = 0; + root.last_context_tokens = root.fullRequestEstimateTokens(); + root.context_local_tokens = root.last_context_tokens; + root.last_cache_read = 0; + } root.fallback_active = !persist; root.fallback_blocked = false; if (persist) saveModel(root.io, root.home, p.id, p.model); @@ -99,13 +112,13 @@ fn applyProviderInner(root: *Agent, arena: Allocator, p: Provider, persist: bool } /// An explicit /model or set_model selection becomes the next-launch default. -pub fn applyProvider(root: *Agent, arena: Allocator, p: Provider) []const u8 { +pub fn applyProvider(root: *Agent, arena: Allocator, p: Provider) ![]const u8 { return applyProviderInner(root, arena, p, true); } /// Automatic failover is session-local: preserve the user's saved preference /// so a repaired credential/model is tried again on the next fresh launch. -pub fn applyFallbackProvider(root: *Agent, arena: Allocator, p: Provider) []const u8 { +pub fn applyFallbackProvider(root: *Agent, arena: Allocator, p: Provider) ![]const u8 { return applyProviderInner(root, arena, p, false); } @@ -182,7 +195,7 @@ pub fn failoverEligible(detail: []const u8) bool { /// credential-backed providers on a clear auth/model/quota failure. Successful /// fallback becomes active for this session but does not replace the saved /// model preference. -pub fn runTurnWithFallback(root: *Agent, keys: Keys, arena: Allocator, out: ?*Io.Writer) anyerror![]const u8 { +pub fn runTurnWithFallback(root: *Agent, keys: *Keys, arena: Allocator, out: ?*Io.Writer) anyerror![]const u8 { if (root.fallback_blocked) return error.FallbackConsentRequired; var attempted: [provider_specs.len][]const u8 = undefined; var attempted_len: usize = 1; @@ -197,10 +210,15 @@ pub fn runTurnWithFallback(root: *Agent, keys: Keys, arena: Allocator, out: ?*Io if (!failoverEligible(detail)) return err; const failed_id = root.provider.id; const failed_model = root.provider.model; - const fallback = nextFallbackProvider(keys, failed_id, attempted[0..attempted_len], root.fallback_allow) orelse return err; + root.ensureStoredKeys(keys); + var fallback = nextFallbackProvider(keys.*, failed_id, attempted[0..attempted_len], root.fallback_allow) orelse return err; + if (std.mem.eql(u8, fallback.id, "codex")) { + root.ensureModelCatalog(keys.*); + fallback = nextFallbackProvider(keys.*, failed_id, attempted[0..attempted_len], root.fallback_allow) orelse return err; + } attempted[attempted_len] = fallback.id; attempted_len += 1; - const note = applyFallbackProvider(root, arena, fallback); + const note = try applyFallbackProvider(root, arena, fallback); if (main_mod.json_mode) { root.emit(.{ .type = "model", .ok = true, .provider = fallback.id, .model = fallback.model, .context = fallback.context, .note = "automatic session fallback; saved model preference kept" }); } else if (out) |w| { @@ -254,6 +272,30 @@ pub fn resolveProviderControlRequest( return resolveProviderRequest(keys, arena, legacy_name); } +/// Whether a human `/model` query can observe account-scoped Codex rows. +/// Explicit non-Codex providers never need the dynamic catalog; bare/fuzzy +/// model queries do because they may name a new account rollout. +pub fn modelQueryMayUseCodex(query: []const u8) bool { + const arg = std.mem.trim(u8, query, " \t"); + if (arg.len == 0) return true; + const provider_end = std.mem.indexOfAny(u8, arg, " /\t") orelse arg.len; + const provider_id = arg[0..provider_end]; + for (provider_specs) |spec| { + if (!std.mem.eql(u8, spec.id, provider_id)) continue; + return std.mem.eql(u8, spec.id, "codex"); + } + return true; +} + +/// Structured set_model has separate provider/model fields. An explicit +/// provider fully determines routing; a model-only request remains fuzzy. +pub fn controlRequestMayUseCodex(provider_query: []const u8, model_query: []const u8, legacy_name: []const u8) bool { + const provider_id = std.mem.trim(u8, provider_query, " \t"); + if (provider_id.len != 0) return std.mem.eql(u8, provider_id, "codex"); + if (std.mem.trim(u8, model_query, " \t").len != 0) return true; + return modelQueryMayUseCodex(legacy_name); +} + fn resolveProviderRequest(keys: *Keys, arena: Allocator, query: []const u8) !Provider { const arg = std.mem.trim(u8, query, " \t"); if (arg.len == 0) return error.InvalidModelRequest; @@ -294,7 +336,7 @@ pub fn setModelRequestLabel(arena: Allocator, provider_query: []const u8, model_ } pub fn switchProvider(root: *Agent, arena: Allocator, p: Provider, out: *Io.Writer) !void { - const note = applyProvider(root, arena, p); + const note = try applyProvider(root, arena, p); try out.print("switched to {s} via {s} ({t} format, {d}k ctx) — {s} · saved for next session\n", .{ p.model, p.id, p.kind, p.context / 1000, note, }); @@ -362,6 +404,61 @@ test "translateHistory: flattens to {role,content:string}, keeps user/assistant, try std.testing.expectEqualStrings("hi", msgs.items[1].object.get("content").?.string); } +test "applyProviderInner preserves the server meter on an exact model re-selection" { + var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator); + defer arena_state.deinit(); + const a = arena_state.allocator(); + const p: Provider = .{ .id = "codex", .kind = .responses, .auth = .bearer, .url = "", .api_key = "", .model = "gpt-5", .context = 270_000 }; + + var root: Agent = undefined; + root.provider = p; + root.arena = a; + root.registry = null; + root.messages = std.json.Array.init(a); + root.sub = false; + root.strict = false; + root.sys_normal = ""; + root.sys_strict = ""; + root.tools_anthropic = ""; + root.tools_openai = ""; + root.tools_responses = ""; + root.keep_context = true; + root.last_context_tokens = 220_000; + root.context_local_tokens = root.fullRequestEstimateTokens(); + root.last_cache_read = 12_345; + root.cap_new = true; + root.effort_rejected = true; + root.ws_off = true; + _ = try applyProviderInner(&root, a, p, false); + try std.testing.expect(root.tools_responses.len > 0); + try std.testing.expectEqual(@as(usize, 0), root.tools_anthropic.len); + try std.testing.expectEqual(@as(usize, 0), root.tools_openai.len); + try std.testing.expectEqual(@as(u64, 220_000), root.last_context_tokens); + try std.testing.expectEqual(@as(u64, 12_345), root.last_cache_read); + try std.testing.expect(root.cap_new); + try std.testing.expect(root.effort_rejected); + try std.testing.expect(root.ws_off); + + var changed = p; + changed.model = "gpt-5.1"; + _ = try applyProviderInner(&root, a, changed, false); + try std.testing.expectEqual(root.fullRequestEstimateTokens(), root.last_context_tokens); + try std.testing.expectEqual(root.last_context_tokens, root.context_local_tokens); + try std.testing.expectEqual(@as(u64, 0), root.last_cache_read); + try std.testing.expect(!root.cap_new); + try std.testing.expect(!root.effort_rejected); + try std.testing.expect(!root.ws_off); + + var changed_format = changed; + changed_format.id = "deepseek"; + changed_format.kind = .openai; + changed_format.model = "deepseek-chat"; + _ = try applyProviderInner(&root, a, changed_format, false); + try std.testing.expect(root.tools_openai.len > 0); + try std.testing.expect(root.tools_responses.len > 0); // already paid for, remains cached + try std.testing.expectEqual(@as(usize, 0), root.tools_anthropic.len); +} + test "set_model control resolves explicit provider/model fields" { var all = Keys{ .values = [_]?[]const u8{"k"} ** provider_specs.len }; var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator); @@ -382,6 +479,19 @@ test "set_model control resolves explicit provider/model fields" { try std.testing.expectEqualStrings("user-loaded/model", local.model); } +test "Codex catalog demand follows model request routing" { + try std.testing.expect(modelQueryMayUseCodex("")); + try std.testing.expect(!modelQueryMayUseCodex("deepseek")); + try std.testing.expect(!modelQueryMayUseCodex("deepseek/deepseek-chat")); + try std.testing.expect(modelQueryMayUseCodex("codex")); + try std.testing.expect(modelQueryMayUseCodex("codex future-sol")); + try std.testing.expect(modelQueryMayUseCodex("future-account-rollout")); + try std.testing.expect(!controlRequestMayUseCodex("codegraff", "deepseek-v4-pro", "")); + try std.testing.expect(controlRequestMayUseCodex("codex", "", "")); + try std.testing.expect(controlRequestMayUseCodex("", "future-account-rollout", "")); + try std.testing.expect(!controlRequestMayUseCodex("", "", "openai")); +} + test "extractText: string content, joined text blocks, and empties" { var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator); defer arena_state.deinit(); diff --git a/src/readline.zig b/src/readline.zig index 2856c3b2..dff25c84 100644 --- a/src/readline.zig +++ b/src/readline.zig @@ -156,17 +156,14 @@ pub fn readLine( out.writeAll("\x1b[6n") catch {}; out.flush() catch {}; dsr: { - var polls: usize = 0; var esc: [16]u8 = undefined; var n: usize = 0; var in_esc = false; while (true) { - if (in.buffered().len == 0 and !inputPending()) { // 50ms poll - polls += 1; - if (polls >= 10) break :dsr; // no reply in ~500ms: fall back - // to column 1 — a later reply is still adopted (CSI 'R'). - continue; - } + // A local terminal answers DSR in a few milliseconds. Do not hold + // every prompt for 500ms when a multiplexer drops/delays it: the + // main loop's CSI 'R' arm adopts a late reply without losing layout. + if (in.buffered().len == 0 and !inputPendingTimed(20)) break :dsr; const b = in.takeByte() catch break :dsr; if (!in_esc) { if (b == 0x1b) { @@ -461,7 +458,7 @@ pub fn readLine( redraw(out, buf.items, cur, marks.items, &rstate, prompt_col); }, 'R' => { // late DSR cursor-position reply (slow or - // multiplexed terminal missed the 500ms startup + // multiplexed terminal missed the 20ms startup // window): adopt the real input column so the // horizontal window stays exact instead of the // column-1 fallback. Same narrow-terminal policy as diff --git a/src/repl.zig b/src/repl.zig index 1189dbc1..66f0e96a 100644 --- a/src/repl.zig +++ b/src/repl.zig @@ -14,7 +14,8 @@ //! //! Split across sibling files (#123, 600-line goal): repl_parser.zig (the //! offline arithmetic evaluator), repl_util.zig (stateless string/format -//! helpers), and repl_model_{turn,commands,render}.zig (the Model struct's +//! helpers), repl_markdown.zig (Markdown and table rendering), and +//! repl_model_{turn,commands,render}.zig (the Model struct's //! method bodies, reached back through the member aliases below via the //! same technique used for the Agent struct split — self.method() and //! Model.method() resolve unchanged regardless of which file backs them). @@ -24,6 +25,7 @@ const zz = @import("zigzag"); const parser_mod = @import("repl_parser.zig"); const util = @import("repl_util.zig"); +const repl_markdown = @import("repl_markdown.zig"); const model_turn = @import("repl_model_turn.zig"); const model_commands = @import("repl_model_commands.zig"); const model_render = @import("repl_model_render.zig"); @@ -32,6 +34,7 @@ const repl_run = @import("repl_run.zig"); test { _ = parser_mod; _ = util; + _ = repl_markdown; _ = model_turn; _ = model_commands; _ = model_render; @@ -106,7 +109,7 @@ pub var g_debug: bool = false; // GRAFF_REPL_DEBUG / `/debug` → dump raw strea // Look & feel. // --------------------------------------------------------------------------- -pub const accent = zz.Color.fromRgb(0xD9, 0x77, 0x57); // Claude coral +pub const accent = util.accent; /// `/ultracode` rainbow shine — sweeps across the input border per frame. pub const rainbow = [_]zz.Color{ @@ -388,381 +391,8 @@ pub const HELP_CALC = \\Offline mode (no model): evaluates i64 arithmetic — + - * / %, parens. ; -/// Render a markdown string to ANSI for display — approximates the harness's -/// streamed renderer: fenced code blocks (left bar), inline `code`, **bold**, -/// # headers, - bullets, and | pipe tables (box-drawing, wrapped to fit -/// `width_hint` columns; 0 = assume 100). Temporaries live in a local arena; -/// the result is owned by `gpa`. -pub fn renderMarkdown(gpa: std.mem.Allocator, src: []const u8, width_hint: usize) ![]const u8 { - var arena_state = std.heap.ArenaAllocator.init(gpa); - defer arena_state.deinit(); - const a = arena_state.allocator(); - var out = std.array_list.Managed(u8).init(a); - - var line_list = std.array_list.Managed([]const u8).init(a); - var it = std.mem.splitScalar(u8, src, '\n'); - while (it.next()) |l| try line_list.append(l); - const lines = line_list.items; - - var in_fence = false; - var first = true; - var idx: usize = 0; - while (idx < lines.len) : (idx += 1) { - const line = lines[idx]; - if (!first) try out.append('\n'); - first = false; - const t = std.mem.trimStart(u8, line, " "); - if (std.mem.startsWith(u8, t, "```")) { - in_fence = !in_fence; - const lang = std.mem.trim(u8, t[3..], " \t"); - const label = if (in_fence and lang.len > 0) lang else "─────"; - try out.appendSlice(try (zz.Style{}).fg(.brightBlack).render(a, label)); - continue; - } - if (in_fence) { - try out.appendSlice(try (zz.Style{}).fg(.brightBlack).render(a, "▏ ")); - try out.appendSlice(try (zz.Style{}).fg(.cyan).render(a, line)); - continue; - } - if (isTableRow(t) and idx + 1 < lines.len and isTableSep(std.mem.trim(u8, lines[idx + 1], " \t"))) { - var end = idx; - while (end < lines.len and isTableRow(std.mem.trimStart(u8, lines[end], " "))) end += 1; - try renderTable(&out, a, lines[idx..end], width_hint); - idx = end - 1; - continue; - } - if (std.mem.startsWith(u8, t, "#")) { - var h = t; - while (h.len > 0 and h[0] == '#') h = h[1..]; - try out.appendSlice(try (zz.Style{}).fg(accent).bold(true).render(a, std.mem.trimStart(u8, h, " "))); - continue; - } - var rest = line; - if (std.mem.startsWith(u8, t, "- ") or std.mem.startsWith(u8, t, "* ")) { - try out.appendSlice(try (zz.Style{}).fg(accent).render(a, " • ")); - rest = t[2..]; - } - try util.renderInline(&out, a, rest); - } - return gpa.dupe(u8, out.items); -} - -fn isTableRow(t: []const u8) bool { - return t.len >= 2 and t[0] == '|'; -} - -/// `|---|:--:|` style alignment row: pipes/colons/spaces only, dashes required. -fn isTableSep(t: []const u8) bool { - if (t.len == 0 or t[0] != '|') return false; - var dash = false; - for (t) |c| switch (c) { - '|', ':', ' ', '\t' => {}, - '-' => dash = true, - else => return false, - }; - return dash; -} - -/// Inline markdown with `code`/**bold** markers stripped — exactly what -/// renderInline makes visible, unstyled. Cell layout is computed from this. -fn plainInline(a: std.mem.Allocator, line: []const u8) ![]const u8 { - var out = std.array_list.Managed(u8).init(a); - var i: usize = 0; - while (i < line.len) { - const c = line[i]; - if (c == '`') { - if (std.mem.indexOfScalarPos(u8, line, i + 1, '`')) |end| { - try out.appendSlice(line[i + 1 .. end]); - i = end + 1; - continue; - } - } else if (c == '*' and i + 1 < line.len and line[i + 1] == '*') { - if (std.mem.indexOfPos(u8, line, i + 2, "**")) |end| { - try out.appendSlice(line[i + 2 .. end]); - i = end + 2; - continue; - } - } - try out.append(c); - i += 1; - } - return out.items; -} - -/// Display columns for one codepoint (wcwidth-style, #142): 0 for combining / -/// zero-width marks, 2 for East-Asian wide + emoji, 1 otherwise. Approximate — -/// covers the ranges that misalign TUI table borders (CJK, Hangul, kana, -/// fullwidth forms, common emoji planes). Emoji-presentation via a VS16 (U+FE0F) -/// on a text-default base is not width-promoted. ASCII/Latin-1 skip the table. -fn codepointWidth(cp: u21) usize { - if (cp < 0x300) return 1; // ASCII + Latin-1 (control bytes stay 1, as before) - if ((cp >= 0x300 and cp <= 0x36F) or // combining diacritical marks - (cp >= 0x200B and cp <= 0x200F) or // ZWSP..RLM - (cp >= 0xFE00 and cp <= 0xFE0F) or // variation selectors - cp == 0xFEFF) return 0; // BOM / ZWNBSP - if ((cp >= 0x1100 and cp <= 0x115F) or // Hangul Jamo - (cp >= 0x2E80 and cp <= 0x303E) or // CJK radicals .. symbols - (cp >= 0x3041 and cp <= 0x33FF) or // kana .. CJK compat - (cp >= 0x3400 and cp <= 0x4DBF) or // CJK ext A - (cp >= 0x4E00 and cp <= 0x9FFF) or // CJK unified - (cp >= 0xA000 and cp <= 0xA4CF) or // Yi - (cp >= 0xAC00 and cp <= 0xD7A3) or // Hangul syllables - (cp >= 0xF900 and cp <= 0xFAFF) or // CJK compat ideographs - (cp >= 0xFE30 and cp <= 0xFE4F) or // CJK compat forms - (cp >= 0xFF00 and cp <= 0xFF60) or // fullwidth forms - (cp >= 0xFFE0 and cp <= 0xFFE6) or // fullwidth signs - (cp >= 0x1F000 and cp <= 0x1F02F) or // mahjong tiles - (cp >= 0x1F0A0 and cp <= 0x1F0FF) or // playing cards - (cp >= 0x1F100 and cp <= 0x1F1FF) or // enclosed alphanumeric / regional - (cp >= 0x1F300 and cp <= 0x1FAFF) or // emoji & pictographs - (cp >= 0x20000 and cp <= 0x3FFFD)) return 2; // CJK ext B+ - return 1; -} - -fn dispWidth(s: []const u8) usize { - var n: usize = 0; - var i: usize = 0; - while (i < s.len) { - const len = std.unicode.utf8ByteSequenceLength(s[i]) catch { - i += 1; - n += 1; - continue; - }; - const end = @min(i + len, s.len); - n += if (std.unicode.utf8Decode(s[i..end])) |cp| codepointWidth(cp) else |_| 1; - i = end; - } - return n; -} - -test "dispWidth: East-Asian wide, combining, emoji (#142)" { - try std.testing.expectEqual(@as(usize, 5), dispWidth("hello")); - try std.testing.expectEqual(@as(usize, 4), dispWidth("\u{4F60}\u{597D}")); // CJK x2 -> 4 cols - try std.testing.expectEqual(@as(usize, 2), dispWidth("\u{3042}")); // hiragana wide - try std.testing.expectEqual(@as(usize, 1), dispWidth("\u{00E9}")); // precomposed e-acute -> 1 - try std.testing.expectEqual(@as(usize, 1), dispWidth("e\u{0301}")); // e + combining acute -> 1 - try std.testing.expectEqual(@as(usize, 2), dispWidth("\u{1F600}")); // emoji -> 2 - try std.testing.expectEqual(@as(usize, 0), dispWidth("\u{200B}")); // ZWSP -> 0 - try std.testing.expectEqual(@as(usize, 2), dispWidth("\u{FF21}")); // fullwidth A -> 2 -} - -/// Word-wrap `s` to `w` display columns; words longer than `w` hard-split at -/// codepoint boundaries. Never emits leading/trailing spaces on a line. -fn wrapCell(a: std.mem.Allocator, s: []const u8, w: usize) ![]const []const u8 { - var lines = std.array_list.Managed([]const u8).init(a); - var cur = std.array_list.Managed(u8).init(a); - var cur_w: usize = 0; - var words = std.mem.tokenizeScalar(u8, s, ' '); - while (words.next()) |word| { - var rem = word; - while (dispWidth(rem) > w) { - if (cur_w > 0) { - try lines.append(try a.dupe(u8, cur.items)); - cur.clearRetainingCapacity(); - cur_w = 0; - } - var bytes: usize = 0; - var cw: usize = 0; - while (bytes < rem.len) { - const clen = @min(std.unicode.utf8ByteSequenceLength(rem[bytes]) catch 1, rem.len - bytes); - const cpw = if (std.unicode.utf8Decode(rem[bytes .. bytes + clen])) |cp| codepointWidth(cp) else |_| 1; - if (cw + cpw > w and bytes > 0) break; // stop before overflowing the column (>=1 char guaranteed) - bytes += clen; - cw += cpw; - } - try lines.append(try a.dupe(u8, rem[0..bytes])); - rem = rem[bytes..]; - } - const ww = dispWidth(rem); - if (ww == 0) continue; - if (cur_w > 0 and cur_w + 1 + ww > w) { - try lines.append(try a.dupe(u8, cur.items)); - cur.clearRetainingCapacity(); - cur_w = 0; - } - if (cur_w > 0) { - try cur.append(' '); - cur_w += 1; - } - try cur.appendSlice(rem); - cur_w += ww; - } - if (cur_w > 0 or lines.items.len == 0) try lines.append(try a.dupe(u8, cur.items)); - return lines.items; -} - -fn tableRule(out: *std.array_list.Managed(u8), a: std.mem.Allocator, widths: []const usize, l: []const u8, m: []const u8, r: []const u8) !void { - var buf = std.array_list.Managed(u8).init(a); - try buf.appendSlice(l); - for (widths, 0..) |w, i| { - for (0..w + 2) |_| try buf.appendSlice("─"); - try buf.appendSlice(if (i + 1 == widths.len) r else m); - } - try out.appendSlice(try (zz.Style{}).fg(.brightBlack).render(a, buf.items)); -} - -/// Narrow-table fallback: one record per data row — bold `Header: value` -/// lines with wrapped continuations indented, a brightBlack rule between -/// records. Used when the box form cannot fit `budget` columns. -fn renderRecords(out: *std.array_list.Managed(u8), a: std.mem.Allocator, rows: []const []const []const u8, budget: usize) !void { - const header = rows[0]; - var rule_buf = std.array_list.Managed(u8).init(a); - const rule_w = @max(@as(usize, 16), @min(budget, 40)); - for (0..rule_w) |_| try rule_buf.appendSlice("─"); - var first_line = true; - for (rows[1..], 0..) |cells, ri| { - if (ri > 0) { - try out.append('\n'); - try out.appendSlice(try (zz.Style{}).fg(.brightBlack).render(a, rule_buf.items)); - } - for (header, 0..) |label, i| { - const value = if (i < cells.len) cells[i] else ""; - const lw = dispWidth(label); - const segs = try wrapCell(a, value, @max(@as(usize, 16), budget -| (lw + 2))); - if (!first_line) try out.append('\n'); - first_line = false; - try out.appendSlice(try (zz.Style{}).bold(true).render(a, label)); - try out.appendSlice(": "); - if (segs.len > 0) try out.appendSlice(segs[0]); - if (segs.len > 1) for (segs[1..]) |seg| { - try out.append('\n'); - try out.appendSlice(" "); - try out.appendSlice(seg); - }; - } - } -} - - -/// Per-column alignment parsed from a table's `:---:` separator row (#143). -const TableAlign = enum { left, center, right }; - -/// Render `| a | b |` source rows (row 1 = alignment separator) as a -/// box-drawing table: bold header, ├─┼─┤ rules between rows, cells -/// word-wrapped so the whole table fits `width_hint` columns. -fn renderTable(out: *std.array_list.Managed(u8), a: std.mem.Allocator, raw_rows: []const []const u8, width_hint: usize) !void { - var rows = std.array_list.Managed([]const []const u8).init(a); - for (raw_rows, 0..) |raw, ri| { - if (ri == 1) continue; // alignment separator row - var body = std.mem.trim(u8, raw, " \t"); - if (body.len > 0 and body[0] == '|') body = body[1..]; - if (body.len > 0 and body[body.len - 1] == '|') body = body[0 .. body.len - 1]; - var cells = std.array_list.Managed([]const u8).init(a); - var cell = std.array_list.Managed(u8).init(a); - var bi: usize = 0; - while (bi < body.len) : (bi += 1) { - if (body[bi] == '\\' and bi + 1 < body.len and body[bi + 1] == '|') { - try cell.append('|'); // escaped \| is a literal pipe, not a column delimiter - bi += 1; - } else if (body[bi] == '|') { - try cells.append(try plainInline(a, std.mem.trim(u8, cell.items, " \t"))); - cell = std.array_list.Managed(u8).init(a); - } else { - try cell.append(body[bi]); - } - } - try cells.append(try plainInline(a, std.mem.trim(u8, cell.items, " \t"))); - try rows.append(cells.items); - } - if (rows.items.len == 0 or rows.items[0].len == 0) return; - const ncols = rows.items[0].len; - - // Column alignment from the separator row (raw_rows[1]): ":--" left, "--:" - // right, ":-:" center, "---" defaults left. Applied to header + data like - // the GUI's ChatTable (#143). Skipped rows never reach here. - const aligns = try a.alloc(TableAlign, ncols); - @memset(aligns, .left); - if (raw_rows.len > 1) { - var sbody = std.mem.trim(u8, raw_rows[1], " \t"); - if (sbody.len > 0 and sbody[0] == '|') sbody = sbody[1..]; - if (sbody.len > 0 and sbody[sbody.len - 1] == '|') sbody = sbody[0 .. sbody.len - 1]; - var ci: usize = 0; - var it = std.mem.splitScalar(u8, sbody, '|'); - while (it.next()) |raw_spec| { - if (ci >= ncols) break; - const spec = std.mem.trim(u8, raw_spec, " \t"); - const lc = spec.len > 0 and spec[0] == ':'; - const rc = spec.len > 0 and spec[spec.len - 1] == ':'; - aligns[ci] = if (lc and rc) .center else if (rc) .right else .left; - ci += 1; - } - } - - const widths = try a.alloc(usize, ncols); - @memset(widths, 1); - for (rows.items) |cells| { - for (cells, 0..) |c, i| { - if (i < ncols) widths[i] = @max(widths[i], dispWidth(c)); - } - } - const budget = (if (width_hint == 0) @as(usize, 100) else @max(width_hint, 40)) -| 8; - const overhead = ncols * 3 + 1; - while (true) { - var total: usize = overhead; - for (widths) |w| total += w; - if (total <= budget) break; - var wi: usize = 0; - var wmax: usize = 0; - for (widths, 0..) |w, i| { - if (w > wmax) { - wmax = w; - wi = i; - } - } - if (wmax <= 8) break; // column floor reached — record fallback below - widths[wi] = wmax - 1; - } - - var total: usize = overhead; - for (widths) |w| total += w; - if (total > budget and rows.items.len >= 2) { - // Even at the column floor the box form overflows the pane — fall back - // to one record per row, mirroring the harness's narrow rendering. - try renderRecords(out, a, rows.items, budget); - return; - } - - try tableRule(out, a, widths, "┌", "┬", "┐"); - for (rows.items, 0..) |cells, ri| { - const wrapped = try a.alloc([]const []const u8, ncols); - var height: usize = 1; - for (0..ncols) |i| { - wrapped[i] = try wrapCell(a, if (i < cells.len) cells[i] else "", widths[i]); - height = @max(height, wrapped[i].len); - } - for (0..height) |li| { - try out.append('\n'); - for (0..ncols) |i| { - try out.appendSlice(try (zz.Style{}).fg(.brightBlack).render(a, "│")); - try out.append(' '); - const seg = if (li < wrapped[i].len) wrapped[i][li] else ""; - const pad = widths[i] -| dispWidth(seg); - const lead: usize = switch (aligns[i]) { - .left => 0, - .right => pad, - .center => pad / 2, - }; - for (0..lead) |_| try out.append(' '); - if (seg.len > 0) { - if (ri == 0) { - try out.appendSlice(try (zz.Style{}).bold(true).render(a, seg)); - } else { - try out.appendSlice(seg); - } - } - for (0..pad - lead) |_| try out.append(' '); - try out.append(' '); - } - try out.appendSlice(try (zz.Style{}).fg(.brightBlack).render(a, "│")); - } - try out.append('\n'); - if (ri + 1 == rows.items.len) { - try tableRule(out, a, widths, "└", "┴", "┘"); - } else { - try tableRule(out, a, widths, "├", "┼", "┤"); - } - } -} +// Markdown and pipe-table rendering lives in repl_markdown.zig. +pub const renderMarkdown = repl_markdown.renderMarkdown; // run/runScripted (the live TUI loop + the headless/scriptable twin) and // main() (the standalone `graff-repl` exe's entry point) live in @@ -964,87 +594,3 @@ test "model: empty Enter force-interrupts when a steer line is queued" { while (m.pending != null and !m.pending.?.done.load(.acquire)) {} if (m.pending != null) m.finishJob(); } - -test "renderMarkdown: pipe table renders as box-drawing" { - const gpa = std.testing.allocator; - const md = "before\n| Impact | Site |\n|---|---|\n| high | `repl.zig:700` |\n| medium | **main.zig** |\nafter"; - const out = try renderMarkdown(gpa, md, 80); - defer gpa.free(out); - try std.testing.expect(std.mem.indexOf(u8, out, "┌") != null); - try std.testing.expect(std.mem.indexOf(u8, out, "┼") != null); - try std.testing.expect(std.mem.indexOf(u8, out, "└") != null); - try std.testing.expect(std.mem.indexOf(u8, out, "repl.zig:700") != null); // backticks stripped in cells - try std.testing.expect(std.mem.indexOf(u8, out, "|---") == null); // separator row consumed - try std.testing.expect(std.mem.indexOf(u8, out, "before") != null); - try std.testing.expect(std.mem.indexOf(u8, out, "after") != null); -} - -test "renderMarkdown: table cells wrap to the width budget" { - const gpa = std.testing.allocator; - var arena_state = std.heap.ArenaAllocator.init(gpa); - defer arena_state.deinit(); - const a = arena_state.allocator(); - const md = "| K | V |\n|---|---|\n| x | this is a very long cell that must wrap across multiple lines to stay inside a narrow table |"; - const out = try renderMarkdown(gpa, md, 48); - defer gpa.free(out); - const plain = util.stripControl(a, out); - var it = std.mem.splitScalar(u8, plain, '\n'); - var cell_rows: usize = 0; - while (it.next()) |line| { - try std.testing.expect(dispWidth(line) <= 48); - if (std.mem.indexOf(u8, line, "│") != null) cell_rows += 1; - } - try std.testing.expect(cell_rows >= 3); // long cell spans several visual lines -} - -test "renderMarkdown: lone pipe line is not a table" { - const gpa = std.testing.allocator; - const out = try renderMarkdown(gpa, "| just text with pipes |\nno separator", 80); - defer gpa.free(out); - try std.testing.expect(std.mem.indexOf(u8, out, "┌") == null); - try std.testing.expect(std.mem.indexOf(u8, out, "| just text with pipes |") != null); -} - -test "renderMarkdown: too-wide table falls back to record layout" { - const gpa = std.testing.allocator; - const md = "| Impact | Site | Finding | Fix safe? |\n|---|---|---|---|\n| high | repl.zig:700 | full scrollback re-styled and re-allocated every single frame | needs care |\n| medium | main.zig:14048 | never-reset session arena grows RSS forever | no |"; - const out = try renderMarkdown(gpa, md, 44); - defer gpa.free(out); - try std.testing.expect(std.mem.indexOf(u8, out, "┌") == null); // box form abandoned - try std.testing.expect(std.mem.indexOf(u8, out, "Impact") != null); // labels repeated per record - try std.testing.expect(std.mem.indexOf(u8, out, "────") != null); // rule between records - try std.testing.expect(std.mem.indexOf(u8, out, "repl.zig:700") != null); - try std.testing.expect(std.mem.indexOf(u8, out, "main.zig:14048") != null); -} - -test "renderMarkdown: escaped pipe stays inside a table cell" { - const gpa = std.testing.allocator; - var arena_state = std.heap.ArenaAllocator.init(gpa); - defer arena_state.deinit(); - const a = arena_state.allocator(); - const md = "| head |\n|---|\n| a \\| b |"; - const out = try renderMarkdown(gpa, md, 100); - defer gpa.free(out); - const plain = util.stripControl(a, out); // arena-owned; freed with arena_state - // the escaped \\| renders as a literal pipe inside the single cell. - try std.testing.expect(std.mem.indexOf(u8, plain, "a | b") != null); -} - -test "renderMarkdown: table honors column alignment" { - const gpa = std.testing.allocator; - var arena_state = std.heap.ArenaAllocator.init(gpa); - defer arena_state.deinit(); - const a = arena_state.allocator(); - // right-aligned: short "5" in the 5-wide "Count" column gets 4 leading spaces. - const rout = try renderMarkdown(gpa, "| Count |\n| ---: |\n| 5 |", 100); - defer gpa.free(rout); - try std.testing.expect(std.mem.indexOf(u8, util.stripControl(a, rout), " 5") != null); - // centered: "x" in the 8-wide "Wide col" column gets leading pad (not flush-left). - const cout = try renderMarkdown(gpa, "| Wide col |\n| :---: |\n| x |", 100); - defer gpa.free(cout); - try std.testing.expect(std.mem.indexOf(u8, util.stripControl(a, cout), " x") != null); - // left default: "5" stays flush-left (no leading pad before it). - const lout = try renderMarkdown(gpa, "| Count |\n| --- |\n| 5 |", 100); - defer gpa.free(lout); - try std.testing.expect(std.mem.indexOf(u8, util.stripControl(a, lout), " 5") == null); -} diff --git a/src/repl_glue.zig b/src/repl_glue.zig index 799de049..c807c272 100644 --- a/src/repl_glue.zig +++ b/src/repl_glue.zig @@ -304,7 +304,7 @@ pub fn replTurnCb(ctx_ptr: ?*anyopaque, gpa: Allocator, history: []const repl.Tu c.fallback_active = agent.fallback_active; c.fallback_blocked = agent.fallback_blocked; } - const final = providers.runTurnWithFallback(&agent, c.keys, arena, &sink.writer) catch |err| switch (err) { + const final = providers.runTurnWithFallback(&agent, &c.keys, arena, &sink.writer) catch |err| switch (err) { // A mid-stream stall (#134): the repl turn IS live (stream_quiet=false), // so postStream can return error.StreamStalled. Don't collapse it to // null — the pane renders that as "model call failed — check /model and diff --git a/src/repl_markdown.zig b/src/repl_markdown.zig new file mode 100644 index 00000000..61b53260 --- /dev/null +++ b/src/repl_markdown.zig @@ -0,0 +1,469 @@ +//! Markdown and pipe-table rendering for the interactive REPL. +//! +//! Kept separate from repl.zig so the REPL facade and Model stay compact. +//! The public entry point remains repl.renderMarkdown. + +const std = @import("std"); +const zz = @import("zigzag"); + +const util = @import("repl_util.zig"); +const accent = util.accent; + +/// Render a markdown string to ANSI for display — approximates the harness's +/// streamed renderer: fenced code blocks (left bar), inline `code`, **bold**, +/// # headers, - bullets, and | pipe tables (box-drawing, wrapped to fit +/// `width_hint` columns; 0 = assume 100). Temporaries live in a local arena; +/// the result is owned by `gpa`. +pub fn renderMarkdown(gpa: std.mem.Allocator, src: []const u8, width_hint: usize) ![]const u8 { + var arena_state = std.heap.ArenaAllocator.init(gpa); + defer arena_state.deinit(); + const a = arena_state.allocator(); + var out = std.array_list.Managed(u8).init(a); + + var line_list = std.array_list.Managed([]const u8).init(a); + var it = std.mem.splitScalar(u8, src, '\n'); + while (it.next()) |l| try line_list.append(l); + const lines = line_list.items; + + var in_fence = false; + var first = true; + var idx: usize = 0; + while (idx < lines.len) : (idx += 1) { + const line = lines[idx]; + if (!first) try out.append('\n'); + first = false; + const t = std.mem.trimStart(u8, line, " "); + if (std.mem.startsWith(u8, t, "```")) { + in_fence = !in_fence; + const lang = std.mem.trim(u8, t[3..], " \t"); + const label = if (in_fence and lang.len > 0) lang else "─────"; + try out.appendSlice(try (zz.Style{}).fg(.brightBlack).render(a, label)); + continue; + } + if (in_fence) { + try out.appendSlice(try (zz.Style{}).fg(.brightBlack).render(a, "▏ ")); + try out.appendSlice(try (zz.Style{}).fg(.cyan).render(a, line)); + continue; + } + if (isTableRow(t) and idx + 1 < lines.len and isTableSep(std.mem.trim(u8, lines[idx + 1], " \t"))) { + var end = idx; + while (end < lines.len and isTableRow(std.mem.trimStart(u8, lines[end], " "))) end += 1; + try renderTable(&out, a, lines[idx..end], width_hint); + idx = end - 1; + continue; + } + if (std.mem.startsWith(u8, t, "#")) { + var h = t; + while (h.len > 0 and h[0] == '#') h = h[1..]; + try out.appendSlice(try (zz.Style{}).fg(accent).bold(true).render(a, std.mem.trimStart(u8, h, " "))); + continue; + } + var rest = line; + if (std.mem.startsWith(u8, t, "- ") or std.mem.startsWith(u8, t, "* ")) { + try out.appendSlice(try (zz.Style{}).fg(accent).render(a, " • ")); + rest = t[2..]; + } + try util.renderInline(&out, a, rest); + } + return gpa.dupe(u8, out.items); +} + +fn isTableRow(t: []const u8) bool { + return t.len >= 2 and t[0] == '|'; +} + +/// `|---|:--:|` style alignment row: pipes/colons/spaces only, dashes required. +fn isTableSep(t: []const u8) bool { + if (t.len == 0 or t[0] != '|') return false; + var dash = false; + for (t) |c| switch (c) { + '|', ':', ' ', '\t' => {}, + '-' => dash = true, + else => return false, + }; + return dash; +} + +/// Inline markdown with `code`/**bold** markers stripped — exactly what +/// renderInline makes visible, unstyled. Cell layout is computed from this. +fn plainInline(a: std.mem.Allocator, line: []const u8) ![]const u8 { + var out = std.array_list.Managed(u8).init(a); + var i: usize = 0; + while (i < line.len) { + const c = line[i]; + if (c == '`') { + if (std.mem.indexOfScalarPos(u8, line, i + 1, '`')) |end| { + try out.appendSlice(line[i + 1 .. end]); + i = end + 1; + continue; + } + } else if (c == '*' and i + 1 < line.len and line[i + 1] == '*') { + if (std.mem.indexOfPos(u8, line, i + 2, "**")) |end| { + try out.appendSlice(line[i + 2 .. end]); + i = end + 2; + continue; + } + } + try out.append(c); + i += 1; + } + return out.items; +} + +/// Display columns for one codepoint (wcwidth-style, #142): 0 for combining / +/// zero-width marks, 2 for East-Asian wide + emoji, 1 otherwise. Approximate — +/// covers the ranges that misalign TUI table borders (CJK, Hangul, kana, +/// fullwidth forms, common emoji planes). Emoji-presentation via a VS16 (U+FE0F) +/// on a text-default base is not width-promoted. ASCII/Latin-1 skip the table. +fn codepointWidth(cp: u21) usize { + if (cp < 0x300) return 1; // ASCII + Latin-1 (control bytes stay 1, as before) + if ((cp >= 0x300 and cp <= 0x36F) or // combining diacritical marks + (cp >= 0x200B and cp <= 0x200F) or // ZWSP..RLM + (cp >= 0xFE00 and cp <= 0xFE0F) or // variation selectors + cp == 0xFEFF) return 0; // BOM / ZWNBSP + if ((cp >= 0x1100 and cp <= 0x115F) or // Hangul Jamo + (cp >= 0x2E80 and cp <= 0x303E) or // CJK radicals .. symbols + (cp >= 0x3041 and cp <= 0x33FF) or // kana .. CJK compat + (cp >= 0x3400 and cp <= 0x4DBF) or // CJK ext A + (cp >= 0x4E00 and cp <= 0x9FFF) or // CJK unified + (cp >= 0xA000 and cp <= 0xA4CF) or // Yi + (cp >= 0xAC00 and cp <= 0xD7A3) or // Hangul syllables + (cp >= 0xF900 and cp <= 0xFAFF) or // CJK compat ideographs + (cp >= 0xFE30 and cp <= 0xFE4F) or // CJK compat forms + (cp >= 0xFF00 and cp <= 0xFF60) or // fullwidth forms + (cp >= 0xFFE0 and cp <= 0xFFE6) or // fullwidth signs + (cp >= 0x1F000 and cp <= 0x1F02F) or // mahjong tiles + (cp >= 0x1F0A0 and cp <= 0x1F0FF) or // playing cards + (cp >= 0x1F100 and cp <= 0x1F1FF) or // enclosed alphanumeric / regional + (cp >= 0x1F300 and cp <= 0x1FAFF) or // emoji & pictographs + (cp >= 0x20000 and cp <= 0x3FFFD)) return 2; // CJK ext B+ + return 1; +} + +fn dispWidth(s: []const u8) usize { + var n: usize = 0; + var i: usize = 0; + while (i < s.len) { + const len = std.unicode.utf8ByteSequenceLength(s[i]) catch { + i += 1; + n += 1; + continue; + }; + const end = @min(i + len, s.len); + n += if (std.unicode.utf8Decode(s[i..end])) |cp| codepointWidth(cp) else |_| 1; + i = end; + } + return n; +} + +test "dispWidth: East-Asian wide, combining, emoji (#142)" { + try std.testing.expectEqual(@as(usize, 5), dispWidth("hello")); + try std.testing.expectEqual(@as(usize, 4), dispWidth("\u{4F60}\u{597D}")); // CJK x2 -> 4 cols + try std.testing.expectEqual(@as(usize, 2), dispWidth("\u{3042}")); // hiragana wide + try std.testing.expectEqual(@as(usize, 1), dispWidth("\u{00E9}")); // precomposed e-acute -> 1 + try std.testing.expectEqual(@as(usize, 1), dispWidth("e\u{0301}")); // e + combining acute -> 1 + try std.testing.expectEqual(@as(usize, 2), dispWidth("\u{1F600}")); // emoji -> 2 + try std.testing.expectEqual(@as(usize, 0), dispWidth("\u{200B}")); // ZWSP -> 0 + try std.testing.expectEqual(@as(usize, 2), dispWidth("\u{FF21}")); // fullwidth A -> 2 +} + +/// Word-wrap `s` to `w` display columns; words longer than `w` hard-split at +/// codepoint boundaries. Never emits leading/trailing spaces on a line. +fn wrapCell(a: std.mem.Allocator, s: []const u8, w: usize) ![]const []const u8 { + var lines = std.array_list.Managed([]const u8).init(a); + var cur = std.array_list.Managed(u8).init(a); + var cur_w: usize = 0; + var words = std.mem.tokenizeScalar(u8, s, ' '); + while (words.next()) |word| { + var rem = word; + while (dispWidth(rem) > w) { + if (cur_w > 0) { + try lines.append(try a.dupe(u8, cur.items)); + cur.clearRetainingCapacity(); + cur_w = 0; + } + var bytes: usize = 0; + var cw: usize = 0; + while (bytes < rem.len) { + const clen = @min(std.unicode.utf8ByteSequenceLength(rem[bytes]) catch 1, rem.len - bytes); + const cpw = if (std.unicode.utf8Decode(rem[bytes .. bytes + clen])) |cp| codepointWidth(cp) else |_| 1; + if (cw + cpw > w and bytes > 0) break; // stop before overflowing the column (>=1 char guaranteed) + bytes += clen; + cw += cpw; + } + try lines.append(try a.dupe(u8, rem[0..bytes])); + rem = rem[bytes..]; + } + const ww = dispWidth(rem); + if (ww == 0) continue; + if (cur_w > 0 and cur_w + 1 + ww > w) { + try lines.append(try a.dupe(u8, cur.items)); + cur.clearRetainingCapacity(); + cur_w = 0; + } + if (cur_w > 0) { + try cur.append(' '); + cur_w += 1; + } + try cur.appendSlice(rem); + cur_w += ww; + } + if (cur_w > 0 or lines.items.len == 0) try lines.append(try a.dupe(u8, cur.items)); + return lines.items; +} + +fn tableRule(out: *std.array_list.Managed(u8), a: std.mem.Allocator, widths: []const usize, l: []const u8, m: []const u8, r: []const u8) !void { + var buf = std.array_list.Managed(u8).init(a); + try buf.appendSlice(l); + for (widths, 0..) |w, i| { + for (0..w + 2) |_| try buf.appendSlice("─"); + try buf.appendSlice(if (i + 1 == widths.len) r else m); + } + try out.appendSlice(try (zz.Style{}).fg(.brightBlack).render(a, buf.items)); +} + +/// Narrow-table fallback: one record per data row — bold `Header: value` +/// lines with wrapped continuations indented, a brightBlack rule between +/// records. Used when the box form cannot fit `budget` columns. +fn renderRecords(out: *std.array_list.Managed(u8), a: std.mem.Allocator, rows: []const []const []const u8, budget: usize) !void { + const header = rows[0]; + var rule_buf = std.array_list.Managed(u8).init(a); + const rule_w = @max(@as(usize, 16), @min(budget, 40)); + for (0..rule_w) |_| try rule_buf.appendSlice("─"); + var first_line = true; + for (rows[1..], 0..) |cells, ri| { + if (ri > 0) { + try out.append('\n'); + try out.appendSlice(try (zz.Style{}).fg(.brightBlack).render(a, rule_buf.items)); + } + for (header, 0..) |label, i| { + const value = if (i < cells.len) cells[i] else ""; + const lw = dispWidth(label); + const segs = try wrapCell(a, value, @max(@as(usize, 16), budget -| (lw + 2))); + if (!first_line) try out.append('\n'); + first_line = false; + try out.appendSlice(try (zz.Style{}).bold(true).render(a, label)); + try out.appendSlice(": "); + if (segs.len > 0) try out.appendSlice(segs[0]); + if (segs.len > 1) for (segs[1..]) |seg| { + try out.append('\n'); + try out.appendSlice(" "); + try out.appendSlice(seg); + }; + } + } +} + +/// Per-column alignment parsed from a table's `:---:` separator row (#143). +const TableAlign = enum { left, center, right }; + +/// Render `| a | b |` source rows (row 1 = alignment separator) as a +/// box-drawing table: bold header, ├─┼─┤ rules between rows, cells +/// word-wrapped so the whole table fits `width_hint` columns. +fn renderTable(out: *std.array_list.Managed(u8), a: std.mem.Allocator, raw_rows: []const []const u8, width_hint: usize) !void { + var rows = std.array_list.Managed([]const []const u8).init(a); + for (raw_rows, 0..) |raw, ri| { + if (ri == 1) continue; // alignment separator row + var body = std.mem.trim(u8, raw, " \t"); + if (body.len > 0 and body[0] == '|') body = body[1..]; + if (body.len > 0 and body[body.len - 1] == '|') body = body[0 .. body.len - 1]; + var cells = std.array_list.Managed([]const u8).init(a); + var cell = std.array_list.Managed(u8).init(a); + var bi: usize = 0; + while (bi < body.len) : (bi += 1) { + if (body[bi] == '\\' and bi + 1 < body.len and body[bi + 1] == '|') { + try cell.append('|'); // escaped \| is a literal pipe, not a column delimiter + bi += 1; + } else if (body[bi] == '|') { + try cells.append(try plainInline(a, std.mem.trim(u8, cell.items, " \t"))); + cell = std.array_list.Managed(u8).init(a); + } else { + try cell.append(body[bi]); + } + } + try cells.append(try plainInline(a, std.mem.trim(u8, cell.items, " \t"))); + try rows.append(cells.items); + } + if (rows.items.len == 0 or rows.items[0].len == 0) return; + const ncols = rows.items[0].len; + + // Column alignment from the separator row (raw_rows[1]): ":--" left, "--:" + // right, ":-:" center, "---" defaults left. Applied to header + data like + // the GUI's ChatTable (#143). Skipped rows never reach here. + const aligns = try a.alloc(TableAlign, ncols); + @memset(aligns, .left); + if (raw_rows.len > 1) { + var sbody = std.mem.trim(u8, raw_rows[1], " \t"); + if (sbody.len > 0 and sbody[0] == '|') sbody = sbody[1..]; + if (sbody.len > 0 and sbody[sbody.len - 1] == '|') sbody = sbody[0 .. sbody.len - 1]; + var ci: usize = 0; + var it = std.mem.splitScalar(u8, sbody, '|'); + while (it.next()) |raw_spec| { + if (ci >= ncols) break; + const spec = std.mem.trim(u8, raw_spec, " \t"); + const lc = spec.len > 0 and spec[0] == ':'; + const rc = spec.len > 0 and spec[spec.len - 1] == ':'; + aligns[ci] = if (lc and rc) .center else if (rc) .right else .left; + ci += 1; + } + } + + const widths = try a.alloc(usize, ncols); + @memset(widths, 1); + for (rows.items) |cells| { + for (cells, 0..) |c, i| { + if (i < ncols) widths[i] = @max(widths[i], dispWidth(c)); + } + } + const budget = (if (width_hint == 0) @as(usize, 100) else @max(width_hint, 40)) -| 8; + const overhead = ncols * 3 + 1; + while (true) { + var total: usize = overhead; + for (widths) |w| total += w; + if (total <= budget) break; + var wi: usize = 0; + var wmax: usize = 0; + for (widths, 0..) |w, i| { + if (w > wmax) { + wmax = w; + wi = i; + } + } + if (wmax <= 8) break; // column floor reached — record fallback below + widths[wi] = wmax - 1; + } + + var total: usize = overhead; + for (widths) |w| total += w; + if (total > budget and rows.items.len >= 2) { + // Even at the column floor the box form overflows the pane — fall back + // to one record per row, mirroring the harness's narrow rendering. + try renderRecords(out, a, rows.items, budget); + return; + } + + try tableRule(out, a, widths, "┌", "┬", "┐"); + for (rows.items, 0..) |cells, ri| { + const wrapped = try a.alloc([]const []const u8, ncols); + var height: usize = 1; + for (0..ncols) |i| { + wrapped[i] = try wrapCell(a, if (i < cells.len) cells[i] else "", widths[i]); + height = @max(height, wrapped[i].len); + } + for (0..height) |li| { + try out.append('\n'); + for (0..ncols) |i| { + try out.appendSlice(try (zz.Style{}).fg(.brightBlack).render(a, "│")); + try out.append(' '); + const seg = if (li < wrapped[i].len) wrapped[i][li] else ""; + const pad = widths[i] -| dispWidth(seg); + const lead: usize = switch (aligns[i]) { + .left => 0, + .right => pad, + .center => pad / 2, + }; + for (0..lead) |_| try out.append(' '); + if (seg.len > 0) { + if (ri == 0) { + try out.appendSlice(try (zz.Style{}).bold(true).render(a, seg)); + } else { + try out.appendSlice(seg); + } + } + for (0..pad - lead) |_| try out.append(' '); + try out.append(' '); + } + try out.appendSlice(try (zz.Style{}).fg(.brightBlack).render(a, "│")); + } + try out.append('\n'); + if (ri + 1 == rows.items.len) { + try tableRule(out, a, widths, "└", "┴", "┘"); + } else { + try tableRule(out, a, widths, "├", "┼", "┤"); + } + } +} + +test "renderMarkdown: pipe table renders as box-drawing" { + const gpa = std.testing.allocator; + const md = "before\n| Impact | Site |\n|---|---|\n| high | `repl.zig:700` |\n| medium | **main.zig** |\nafter"; + const out = try renderMarkdown(gpa, md, 80); + defer gpa.free(out); + try std.testing.expect(std.mem.indexOf(u8, out, "┌") != null); + try std.testing.expect(std.mem.indexOf(u8, out, "┼") != null); + try std.testing.expect(std.mem.indexOf(u8, out, "└") != null); + try std.testing.expect(std.mem.indexOf(u8, out, "repl.zig:700") != null); // backticks stripped in cells + try std.testing.expect(std.mem.indexOf(u8, out, "|---") == null); // separator row consumed + try std.testing.expect(std.mem.indexOf(u8, out, "before") != null); + try std.testing.expect(std.mem.indexOf(u8, out, "after") != null); +} + +test "renderMarkdown: table cells wrap to the width budget" { + const gpa = std.testing.allocator; + var arena_state = std.heap.ArenaAllocator.init(gpa); + defer arena_state.deinit(); + const a = arena_state.allocator(); + const md = "| K | V |\n|---|---|\n| x | this is a very long cell that must wrap across multiple lines to stay inside a narrow table |"; + const out = try renderMarkdown(gpa, md, 48); + defer gpa.free(out); + const plain = util.stripControl(a, out); + var it = std.mem.splitScalar(u8, plain, '\n'); + var cell_rows: usize = 0; + while (it.next()) |line| { + try std.testing.expect(dispWidth(line) <= 48); + if (std.mem.indexOf(u8, line, "│") != null) cell_rows += 1; + } + try std.testing.expect(cell_rows >= 3); // long cell spans several visual lines +} + +test "renderMarkdown: lone pipe line is not a table" { + const gpa = std.testing.allocator; + const out = try renderMarkdown(gpa, "| just text with pipes |\nno separator", 80); + defer gpa.free(out); + try std.testing.expect(std.mem.indexOf(u8, out, "┌") == null); + try std.testing.expect(std.mem.indexOf(u8, out, "| just text with pipes |") != null); +} + +test "renderMarkdown: too-wide table falls back to record layout" { + const gpa = std.testing.allocator; + const md = "| Impact | Site | Finding | Fix safe? |\n|---|---|---|---|\n| high | repl.zig:700 | full scrollback re-styled and re-allocated every single frame | needs care |\n| medium | main.zig:14048 | never-reset session arena grows RSS forever | no |"; + const out = try renderMarkdown(gpa, md, 44); + defer gpa.free(out); + try std.testing.expect(std.mem.indexOf(u8, out, "┌") == null); // box form abandoned + try std.testing.expect(std.mem.indexOf(u8, out, "Impact") != null); // labels repeated per record + try std.testing.expect(std.mem.indexOf(u8, out, "────") != null); // rule between records + try std.testing.expect(std.mem.indexOf(u8, out, "repl.zig:700") != null); + try std.testing.expect(std.mem.indexOf(u8, out, "main.zig:14048") != null); +} + +test "renderMarkdown: escaped pipe stays inside a table cell" { + const gpa = std.testing.allocator; + var arena_state = std.heap.ArenaAllocator.init(gpa); + defer arena_state.deinit(); + const a = arena_state.allocator(); + const md = "| head |\n|---|\n| a \\| b |"; + const out = try renderMarkdown(gpa, md, 100); + defer gpa.free(out); + const plain = util.stripControl(a, out); // arena-owned; freed with arena_state + // the escaped \\| renders as a literal pipe inside the single cell. + try std.testing.expect(std.mem.indexOf(u8, plain, "a | b") != null); +} + +test "renderMarkdown: table honors column alignment" { + const gpa = std.testing.allocator; + var arena_state = std.heap.ArenaAllocator.init(gpa); + defer arena_state.deinit(); + const a = arena_state.allocator(); + // right-aligned: short "5" in the 5-wide "Count" column gets 4 leading spaces. + const rout = try renderMarkdown(gpa, "| Count |\n| ---: |\n| 5 |", 100); + defer gpa.free(rout); + try std.testing.expect(std.mem.indexOf(u8, util.stripControl(a, rout), " 5") != null); + // centered: "x" in the 8-wide "Wide col" column gets leading pad (not flush-left). + const cout = try renderMarkdown(gpa, "| Wide col |\n| :---: |\n| x |", 100); + defer gpa.free(cout); + try std.testing.expect(std.mem.indexOf(u8, util.stripControl(a, cout), " x") != null); + // left default: "5" stays flush-left (no leading pad before it). + const lout = try renderMarkdown(gpa, "| Count |\n| --- |\n| 5 |", 100); + defer gpa.free(lout); + try std.testing.expect(std.mem.indexOf(u8, util.stripControl(a, lout), " 5") == null); +} diff --git a/src/repl_util.zig b/src/repl_util.zig index cde9fcda..88e6f87c 100644 --- a/src/repl_util.zig +++ b/src/repl_util.zig @@ -6,6 +6,8 @@ const std = @import("std"); const zz = @import("zigzag"); +pub const accent = zz.Color.fromRgb(0xD9, 0x77, 0x57); // Claude coral + pub const HELP_CHAT = \\Commands (mirrors the graff session): \\ /help /clear /new /quit conversation diff --git a/src/schema.zig b/src/schema.zig index b735faf9..16d8474f 100644 --- a/src/schema.zig +++ b/src/schema.zig @@ -216,6 +216,21 @@ const workflow_spec = ToolSpec{ pub const root_specs = base_specs ++ meta_specs ++ [_]ToolSpec{ subagent_spec, workflow_spec }; +// The common catalog (clock_sleep off) is static too. Previously every root +// startup allocated and filled a filtered ToolSpec array before rendering even +// one provider catalog. +const root_specs_without_clock = blk: { + var out: [root_specs.len - 1]ToolSpec = undefined; + var len: usize = 0; + for (root_specs) |tool| { + if (std.mem.eql(u8, tool.name, "clock_sleep")) continue; + out[len] = tool; + len += 1; + } + if (len != out.len) @compileError("root tool catalog must contain exactly one clock_sleep entry"); + break :blk out; +}; + pub fn isMetaName(name: []const u8) bool { return std.mem.eql(u8, name, "todo_write") or std.mem.eql(u8, name, "todo_read") or @@ -292,12 +307,9 @@ pub fn renderRootTools( /// build their tool lists from base_specs only (tools_*_sub above), which /// never included meta_specs/clock_sleep in the first place. pub fn effectiveRootSpecs(arena: Allocator) ![]const ToolSpec { + _ = arena; if (root.g_clock_sleep) return &root_specs; - var out: std.ArrayList(ToolSpec) = .empty; - for (root_specs) |t| { - if (!std.mem.eql(u8, t.name, "clock_sleep")) try out.append(arena, t); - } - return out.items; + return &root_specs_without_clock; } const Schema = union(enum) { raw: []const u8, value: Value }; @@ -504,6 +516,7 @@ test "effectiveRootSpecs: drops clock_sleep from the root tool catalog unless th const off_specs = try effectiveRootSpecs(a); try std.testing.expectEqual(root_specs.len - 1, off_specs.len); for (off_specs) |t| try std.testing.expect(!std.mem.eql(u8, t.name, "clock_sleep")); + try std.testing.expectEqual(@as(usize, 0), arena_state.queryCapacity()); root.g_clock_sleep = true; const on_specs = try effectiveRootSpecs(a); diff --git a/src/serde.zig b/src/serde.zig index 569e3409..03c88076 100644 --- a/src/serde.zig +++ b/src/serde.zig @@ -14,6 +14,8 @@ const util = @import("util.zig"); const history_file = ".simple-harness-history"; const model_file = ".simple-harness-model"; // remembers the last-selected provider+model +pub const SavedModel = struct { pid: []const u8, model: []const u8 }; + /// Persist the active provider+model so the next launch starts on it. pub fn saveModel(io: Io, home: []const u8, pid: []const u8, model: []const u8) void { if (home.len == 0) return; @@ -28,7 +30,7 @@ pub fn saveModel(io: Io, home: []const u8, pid: []const u8, model: []const u8) v } /// Load the remembered provider+model (pid on line 1, model on line 2). -pub fn loadModel(io: Io, arena: Allocator, home: []const u8) ?struct { pid: []const u8, model: []const u8 } { +pub fn loadModel(io: Io, arena: Allocator, home: []const u8) ?SavedModel { if (home.len == 0) return null; const path = std.fmt.allocPrint(arena, "{s}/{s}", .{ home, model_file }) catch return null; const data = Io.Dir.cwd().readFileAlloc(io, path, arena, .limited(4096)) catch return null; diff --git a/src/session.zig b/src/session.zig index 14771954..f37f5664 100644 --- a/src/session.zig +++ b/src/session.zig @@ -246,7 +246,24 @@ pub fn saveSession(root: *Agent, arena: Allocator, name: []const u8) !void { try s.objectField("updated_ms"); try s.write(unixMs(root.io)); try s.objectField("messages"); + const messages_start = aw.writer.buffered().len; try s.write(Value{ .array = root.messages }); + const messages_bytes = aw.writer.buffered().len - messages_start; + const context_estimate = root.contextEstimateFromInputBytes(messages_bytes); + // Preserve the last authoritative provider reading across resume. Responses + // histories can contain compact encrypted-reasoning handles whose serialized + // byte size substantially underestimates their server-side token cost; without + // this floor a resumed near-limit session can miss pre-send compaction. + try s.objectField("context_tokens"); + // std.json.Value represents parsed integers as i64, so keep the persisted + // value inside the range loadSession can round-trip even if a malformed + // provider once reported an extreme unsigned count. + try s.write(@min(context_estimate.effective, @as(u64, std.math.maxInt(i64)))); + // Pair the provider reading with the locally measurable request component. + // On resume we can preserve only their hidden-token delta while replacing + // this component with today's prompt/tool-schema estimate. + try s.objectField("context_local_tokens"); + try s.write(@min(context_estimate.local, @as(u64, std.math.maxInt(i64)))); try s.endObject(); Io.Dir.cwd().createDir(root.io, ".graff", .default_dir) catch {}; @@ -305,10 +322,39 @@ test "goalFromValue: legacy string -> active; object round-trips; paused stays p const unknown = try std.json.parseFromSliceLeaky(Value, a, "{\"objective\":\"x\",\"status\":\"zzz\"}", .{ .allocate = .alloc_always }); try std.testing.expectEqual(agent_mod.GoalStatus.active, goalFromValue(unknown, 1).?.status); } + +fn contextTokensFromSession(obj: std.json.ObjectMap) u64 { + const v = obj.get("context_tokens") orelse return 0; + if (v != .integer or v.integer <= 0) return 0; + return @intCast(v.integer); +} + +fn contextLocalTokensFromSession(obj: std.json.ObjectMap) ?u64 { + const v = obj.get("context_local_tokens") orelse return null; + if (v != .integer or v.integer < 0) return null; + return @intCast(v.integer); +} + +fn restoreContextMeter(root: *Agent, saved_context_tokens: u64, saved_local_tokens: ?u64) void { + const current_local = root.fullRequestEstimateTokens(); + // A provider reading is meaningful across resume only after separating its + // locally measurable component. Preserve the server-only delta (encrypted + // reasoning, tokenizer differences, output usage), but replace the old + // prompt/tool-schema estimate with the current one. Files predating this + // paired field safely re-estimate instead of trusting an ungrounded meter. + const hidden_delta = if (saved_local_tokens) |saved_local| + saved_context_tokens -| saved_local + else + 0; + root.last_context_tokens = current_local +| hidden_delta; + root.context_local_tokens = current_local; + root.last_cache_read = 0; +} + /// Restore a saved session: parse the file (arena-owned), rebuild the /// provider, and replace the live history. The wire format must still match /// the restored provider's kind — same provider id guarantees it. -pub fn loadSession(root: *Agent, keys: Keys, arena: Allocator, name: []const u8) !void { +pub fn loadSession(root: *Agent, keys: *Keys, arena: Allocator, name: []const u8) !void { const path = try sessionPath(arena, name); const data = Io.Dir.cwd().readFileAlloc(root.io, path, arena, .limited(8 * 1024 * 1024)) catch blk: { // backward-compat: older builds wrote .session.json in cwd. @@ -325,8 +371,19 @@ pub fn loadSession(root: *Agent, keys: Keys, arena: Allocator, name: []const u8) const ultracode_mode = if (obj.get("ultracode_mode")) |v| (v == .bool and v.bool) else false; const goal: ?agent_mod.Goal = if (obj.get("goal")) |v| goalFromValue(v, unixMs(root.io)) else null; const title = if (obj.get("title")) |v| (if (v == .string and v.string.len > 0) v.string else null) else null; + // Optional for backward compatibility with sessions written before context + // metering was persisted. JSON integers are signed; ignore negative/wrong-type + // values instead of turning a corrupt session into an enormous unsigned count. + const saved_context_tokens = contextTokensFromSession(obj); + const saved_local_tokens = contextLocalTokensFromSession(obj); + root.ensureStoredKeys(keys); + if (std.mem.eql(u8, pid, "codex")) root.ensureModelCatalog(keys.*); root.provider = try keys.providerById(pid, model); + // A resumed session may use a different wire format than the startup + // default. Materialize that catalog before rebasing its saved context + // meter, and keep every still-unused format lazy. + try root.ensureRootTools(root.provider.kind); root.messages = msgs; // Repair histories written by older builds where a Responses // `function_call_output.output` was persisted as a byte array instead of a @@ -353,9 +410,81 @@ pub fn loadSession(root: *Agent, keys: Keys, arena: Allocator, name: []const u8) root.ultracode_mode = ultracode_mode; root.goal = goal; root.session_title = title; - root.last_context_tokens = 0; + // Rebase the saved server-only delta onto today's prompt/tool-schema input. + restoreContextMeter(root, saved_context_tokens, saved_local_tokens); root.cap_new = false; // per-provider; relearn on rejection root.effort_rejected = false; + root.ws_off = false; // transport failures belong to the prior live session + root.compact_transport_failures = 0; + root.last_usage_includes_output = false; + root.last_request_context_overflow = false; + root.last_request_write_failed = false; + root.fallback_active = false; + root.fallback_blocked = false; +} + +test "session context meter restores hidden delta and legacy sessions re-estimate" { + var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator); + defer arena_state.deinit(); + const a = arena_state.allocator(); + + const current = try std.json.parseFromSliceLeaky(Value, a, "{\"context_tokens\":90000,\"context_local_tokens\":20000}", .{}); + try std.testing.expectEqual(@as(u64, 90_000), contextTokensFromSession(current.object)); + try std.testing.expectEqual(@as(u64, 20_000), contextLocalTokensFromSession(current.object).?); + const legacy = try std.json.parseFromSliceLeaky(Value, a, "{}", .{}); + try std.testing.expectEqual(@as(u64, 0), contextTokensFromSession(legacy.object)); + try std.testing.expect(contextLocalTokensFromSession(legacy.object) == null); + const invalid = try std.json.parseFromSliceLeaky(Value, a, "{\"context_tokens\":-1}", .{}); + try std.testing.expectEqual(@as(u64, 0), contextTokensFromSession(invalid.object)); + const invalid_local = try std.json.parseFromSliceLeaky(Value, a, "{\"context_local_tokens\":-1}", .{}); + try std.testing.expect(contextLocalTokensFromSession(invalid_local.object) == null); + + // Model a resumed Responses history whose compact encrypted-reasoning handle + // serializes much smaller than the authoritative server token reading. + var msgs = std.json.Array.init(a); + const reasoning = "{\"type\":\"reasoning\",\"encrypted_content\":\"" ++ ("x" ** 8192) ++ "\"}"; + try msgs.append(try std.json.parseFromSliceLeaky(Value, a, reasoning, .{})); + var root: Agent = undefined; + root.provider = .{ .id = "codex", .kind = .responses, .auth = .bearer, .url = "", .api_key = "", .model = "gpt-5", .context = 100_000 }; + root.messages = msgs; + root.sub = false; + root.strict = false; + root.sys_normal = ""; + root.sys_strict = ""; + root.tools_responses = ""; + root.last_cache_read = 12_345; + + const local_estimate = root.fullRequestEstimateTokens(); + try std.testing.expect(local_estimate < 90_000); + restoreContextMeter(&root, contextTokensFromSession(current.object), contextLocalTokensFromSession(current.object)); + try std.testing.expectEqual(local_estimate + 70_000, root.last_context_tokens); + try std.testing.expectEqual(@as(u64, 0), root.last_cache_read); + try std.testing.expect(root.last_context_tokens > local_estimate); + + // A legitimate over-window reading remains evidence when its paired local + // estimate matches the current request. + restoreContextMeter(&root, 150_000, local_estimate); + try std.testing.expectEqual(@as(u64, 150_000), root.last_context_tokens); + + // An unknown window still preserves the paired hidden delta; no clamping is + // needed or safe. + root.provider.context = 0; + restoreContextMeter(&root, 90_000, local_estimate); + try std.testing.expectEqual(root.fullRequestEstimateTokens() + (90_000 - local_estimate), root.last_context_tokens); + root.provider.context = 100_000; + + // A legacy session has no saved meter, but non-empty restored history must + // still produce a live local reading rather than resetting to zero. + root.last_cache_read = 99; + restoreContextMeter(&root, contextTokensFromSession(legacy.object), contextLocalTokensFromSession(legacy.object)); + try std.testing.expectEqual(local_estimate, root.last_context_tokens); + try std.testing.expectEqual(@as(u64, 0), root.last_cache_read); + + // A meter from an intermediate build without the paired local component is + // also ungrounded and must not force destructive recovery after resume. + const unpaired = try std.json.parseFromSliceLeaky(Value, a, "{\"context_tokens\":99000}", .{}); + restoreContextMeter(&root, contextTokensFromSession(unpaired.object), contextLocalTokensFromSession(unpaired.object)); + try std.testing.expectEqual(local_estimate, root.last_context_tokens); } test "slugifyTitle makes a filesystem-safe slug from an AI title" { diff --git a/src/session_run.zig b/src/session_run.zig index 7c9113e1..724ccd2a 100644 --- a/src/session_run.zig +++ b/src/session_run.zig @@ -52,7 +52,6 @@ const messages_mod = @import("messages.zig"); const session = @import("session.zig"); const fleet = @import("fleet.zig"); const hooks = @import("hooks.zig"); -const schema = @import("schema.zig"); /// `graff repl`: interactive chat REPL on the zigzag TUI, backed by the REAL /// agent loop — each prompt runs a full root turn (tools + MCP) via @@ -61,12 +60,19 @@ const schema = @import("schema.zig"); /// `root` is already a stable, fully-constructed main()-owned Agent by the /// time this is called, so taking its address here is safe (this helper /// only reads through the pointer, it never owns or returns Agent storage). -pub fn runReplCommand(gpa: Allocator, io: Io, environ_map: anytype, root: *agent_mod.Agent, keys: provider_mod.Keys, client: *std.http.Client, in: *Io.Reader, out: *Io.Writer, arena: Allocator, flags: args.Flags) !bool { +pub fn runReplCommand(gpa: Allocator, io: Io, environ_map: anytype, root: *agent_mod.Agent, keys: *provider_mod.Keys, client: *std.http.Client, in: *Io.Reader, out: *Io.Writer, arena: Allocator, flags: args.Flags) !bool { if (!(flags.positionals.items.len > 0 and std.mem.eql(u8, flags.positionals.items[0], "repl"))) return false; + root.ensureStoredKeys(keys); + root.ensureModelCatalog(keys.*); + // The standalone chat REPL can switch wire formats inside its own model + // picker, so materialize every catalog only when this surface is entered. + try root.ensureRootTools(.anthropic); + try root.ensureRootTools(.openai); + try root.ensureRootTools(.responses); var repl_ctx = repl_glue.ReplCtx{ .io = io, .client = client, - .keys = keys, + .keys = keys.*, .home = root.home, .provider = root.provider, .fallback_allow = root.fallback_allow, @@ -97,7 +103,7 @@ pub fn runReplCommand(gpa: Allocator, io: Io, environ_map: anytype, root: *agent /// denies anything not pre-approved instead of prompting (there's no one to /// ask). Moved out of main() verbatim (600-line goal); `root`/`tracer` are /// already stable main()-owned storage by the time this runs. -pub fn runOneshotPrompt(gpa: Allocator, io: Io, arena: Allocator, root: *agent_mod.Agent, keys: provider_mod.Keys, tracer: *trace.Tracer, out: *Io.Writer, prompt_text: []const u8) !void { +pub fn runOneshotPrompt(gpa: Allocator, io: Io, arena: Allocator, root: *agent_mod.Agent, keys: *provider_mod.Keys, tracer: *trace.Tracer, out: *Io.Writer, prompt_text: []const u8) !void { main_mod.unattended = true; root.in = null; // gate: deny instead of prompt; ask_user: self-decide root.out = null; // tool progress → stderr; stdout carries only the answer @@ -195,14 +201,10 @@ pub fn buildRootAgent( tracer: *trace.Tracer, sys_normal: []const u8, sys_strict: []const u8, - mcp_tools: []const mcp.Tool, snaps: *tools_mod.Snapshots, flags: args.Flags, telem_endpoint: []const u8, ) !agent_mod.Agent { - // #225: clock_sleep only appears in the model's tool catalog when the - // root-only feature flag is on (--clock-sleep / GRAFF_CLOCK_SLEEP=1). - const root_tool_specs = try schema.effectiveRootSpecs(arena); var root: agent_mod.Agent = .{ .snapshots = snaps, .gpa = gpa, @@ -221,10 +223,13 @@ pub fn buildRootAgent( .tracer = tracer, .sys_normal = sys_normal, .sys_strict = sys_strict, - .tools_anthropic = try schema.renderRootTools(arena, .anthropic, root_tool_specs, mcp_tools), - .tools_openai = try schema.renderRootTools(arena, .openai, root_tool_specs, mcp_tools), - .tools_responses = try schema.renderRootTools(arena, .responses, root_tool_specs, mcp_tools), + .tools_anthropic = "", + .tools_openai = "", + .tools_responses = "", }; + // Startup pays for one provider format, not all three. Other formats are + // rendered on first switch with the same built-in + live MCP inputs. + try root.ensureRootTools(default_provider.kind); const fresh_session_name = try std.fmt.allocPrint(arena, "session-{d}", .{util.unixMs(io)}); root.session_name = if (flags.resume_flag) |name| (if (!flags.new_session_flag and !flags.no_resume_flag) name else fresh_session_name) else fresh_session_name; repl_glue.loadThinkingSettings(io, arena, &root); // {"effort":...,"fast":...} persisted by /effort and /fast @@ -251,7 +256,7 @@ pub fn buildRootAgent( /// resume-target reload for a resumed one-shot. Moved out of main() verbatim /// (600-line goal). Both are best-effort (`catch {}`), matching main()'s /// former inline behavior exactly. -pub fn saveOrResumeSession(root: *agent_mod.Agent, keys: provider_mod.Keys, arena: Allocator, flags: args.Flags) void { +pub fn saveOrResumeSession(root: *agent_mod.Agent, keys: *provider_mod.Keys, arena: Allocator, flags: args.Flags) void { const will_resume = flags.resume_flag != null and !flags.new_session_flag and !flags.no_resume_flag; if (!will_resume) session.saveSession(root, arena, root.session_name) catch {}; if (flags.oneshot_prompt != null and flags.resume_flag != null and !flags.new_session_flag and !flags.no_resume_flag) { @@ -265,13 +270,10 @@ pub fn saveOrResumeSession(root: *agent_mod.Agent, keys: provider_mod.Keys, aren /// as what would trigger live compaction, summarizes up front instead of /// re-billing the whole thing on the first turn. Moved out of main() /// verbatim (600-line goal); `root` is already stable main()-owned storage. -pub fn restoreResumedSession(io: Io, arena: Allocator, out: *Io.Writer, root: *agent_mod.Agent, keys: provider_mod.Keys, flags: args.Flags, json_mode: bool, cwd_display: []const u8) !void { +pub fn restoreResumedSession(arena: Allocator, out: *Io.Writer, root: *agent_mod.Agent, keys: *provider_mod.Keys, flags: args.Flags, json_mode: bool, cwd_display: []const u8) !void { if (!(flags.oneshot_prompt == null and flags.resume_flag != null and !flags.new_session_flag and !flags.no_resume_flag)) return; if (session.loadSession(root, keys, arena, root.session_name)) |_| { if (root.messages.items.len > 0) { - // Estimate the restored context from the file size (~4 bytes/token). - const est_path = try session.sessionPath(arena, root.session_name); - const est: u64 = if (Io.Dir.cwd().statFile(io, est_path, .{})) |st| @as(u64, @intCast(st.size)) / 4 else |_| 0; if (!json_mode) { // Prefer the saved AI summary; fall back to the first user // message only for older sessions that have no saved title. @@ -282,12 +284,13 @@ pub fn restoreResumedSession(io: Io, arena: Allocator, out: *Io.Writer, root: *a try out.print("↩ resumed {s}{s} — {d} message(s) on {s} · /new or /clear for a fresh start\n", .{ root.session_name, session.session_ext, root.messages.items.len, root.provider.model }); try out.flush(); } - // Cold cache: if the restored context is as large as what would - // trigger live compaction, the first turn would re-bill the whole - // thing — summarize up front instead. - if (est >= root.provider.compactAt()) { - root.last_context_tokens = est; - root.compactOrRecover(true); + // Cold cache: loadSession rebased the saved server-only token delta + // onto today's complete local request estimate. Summarize up front + // when that conservative meter crosses compactAt. Resume itself is + // never permission to destructively trim on a transient/empty + // summary; a concrete provider overflow can still override this. + if (root.inputOverCompactThreshold()) { + root.compactOrRecover(false); } } } else |_| {} diff --git a/src/startup.zig b/src/startup.zig index 8c01186d..993c7577 100644 --- a/src/startup.zig +++ b/src/startup.zig @@ -43,8 +43,98 @@ pub const ResolvedKeys = struct { stale_saved_model: ?[]const u8, preferred_provider: ?[]const u8, codex_account: ?[]const u8, + model_catalog: models_cache.LazyCodexCatalog, + stored_keys_loaded: bool, }; +fn explicitProvider(model_flag: ?[]const u8) ?[]const u8 { + const query = model_flag orelse return null; + for (provider_mod.provider_specs) |spec| + if (std.mem.eql(u8, spec.id, query)) return spec.id; + return null; +} + +/// Whether one stored credential can affect an explicit startup selection. +/// Provider ids need one key; exact catalog model names need only providers +/// that serve that model. Aliases/fuzzy queries retain the full scan because +/// key availability participates in their tie-break. +fn storedKeyMayAffectSelection(provider_id: []const u8, model_flag: ?[]const u8) bool { + const query = model_flag orelse return true; + if (explicitProvider(query)) |id| return std.mem.eql(u8, provider_id, id); + if (pricing.modelInTable(query)) return pricing.providerModelInTable(provider_id, query); + return true; +} + +fn hasSelectiveStoredKeyScope(model_flag: ?[]const u8) bool { + const query = model_flag orelse return false; + return explicitProvider(query) != null or pricing.modelInTable(query); +} + +fn startupStoredKeyScope(model_flag: ?[]const u8, selective: bool) keys_cli.StoredKeyScope { + if (!selective) return .all; + var mask = [_]bool{false} ** provider_mod.provider_specs.len; + for (provider_mod.provider_specs, 0..) |spec, i| + mask[i] = storedKeyMayAffectSelection(spec.id, model_flag); + return .{ .mask = mask }; +} + +fn startupNeedsCodexCatalog(keys: provider_mod.Keys, model_flag: ?[]const u8, saved: ?serde.SavedModel) bool { + if (keys.get("codex") == null) return false; + if (model_flag) |query| { + for (provider_mod.provider_specs) |spec| if (std.mem.eql(u8, spec.id, query)) + return std.mem.eql(u8, spec.id, "codex"); + // A free-form query may name an account-only rollout. + return true; + } + if (saved) |selection| { + if (std.mem.eql(u8, selection.pid, "codex")) return true; + if (pricing.providerModelInTable(selection.pid, selection.model) and keys.get(selection.pid) != null) return false; + // A stale/keyless preference may fall through to Codex. + return true; + } + const fallback = keys.defaultProvider() catch return false; + return std.mem.eql(u8, fallback.id, "codex"); +} + +test "Codex catalog loads at startup only when selection can observe it" { + try std.testing.expectEqualStrings("deepseek", explicitProvider("deepseek").?); + try std.testing.expect(explicitProvider("deepseek-v4-pro") == null); + try std.testing.expect(explicitProvider(null) == null); + try std.testing.expect(hasSelectiveStoredKeyScope("deepseek-v4-pro")); + try std.testing.expect(storedKeyMayAffectSelection("codegraff", "deepseek-v4-pro")); + try std.testing.expect(storedKeyMayAffectSelection("deepseek", "deepseek-v4-pro")); + try std.testing.expect(!storedKeyMayAffectSelection("anthropic", "deepseek-v4-pro")); + try std.testing.expect(storedKeyMayAffectSelection("anthropic", "opus")); + try std.testing.expect(storedKeyMayAffectSelection("openai", null)); + const exact_scope = startupStoredKeyScope("deepseek-v4-pro", true); + try std.testing.expect(exact_scope.includes(1, "codegraff")); + try std.testing.expect(exact_scope.includes(2, "deepseek")); + try std.testing.expect(!exact_scope.includes(0, "anthropic")); + + var keys: provider_mod.Keys = .{ .values = [_]?[]const u8{null} ** provider_mod.provider_specs.len }; + for (provider_mod.provider_specs, &keys.values) |spec, *value| { + if (std.mem.eql(u8, spec.id, "deepseek") or std.mem.eql(u8, spec.id, "codex")) value.* = "test"; + } + keys.codex_account = "account"; + + try std.testing.expect(!startupNeedsCodexCatalog(keys, "deepseek", null)); + try std.testing.expect(startupNeedsCodexCatalog(keys, "codex", null)); + try std.testing.expect(startupNeedsCodexCatalog(keys, "future-account-rollout", null)); + try std.testing.expect(!startupNeedsCodexCatalog(keys, null, .{ .pid = "deepseek", .model = "deepseek-v4-pro" })); + try std.testing.expect(!startupNeedsCodexCatalog(keys, null, null)); + + for (provider_mod.provider_specs, &keys.values) |spec, *value| { + if (std.mem.eql(u8, spec.id, "deepseek")) value.* = null; + } + try std.testing.expect(startupNeedsCodexCatalog(keys, null, null)); + try std.testing.expect(startupNeedsCodexCatalog(keys, null, .{ .pid = "deepseek", .model = "deepseek-v4-pro" })); + + for (provider_mod.provider_specs, &keys.values) |spec, *value| { + if (std.mem.eql(u8, spec.id, "codex")) value.* = null; + } + try std.testing.expect(!startupNeedsCodexCatalog(keys, "codex", null)); +} + /// Resolves API keys/credentials (env vars → codegraff/codex/kimi on-disk /// logins → the `harness key set` store, env always wins — same precedence /// as before) and picks the startup model (--model flag, else the @@ -108,36 +198,35 @@ pub fn resolveKeys(io: Io, gpa: Allocator, arena: Allocator, environ_map: anytyp } } } - // Stored keys (macOS Keychain / 0600 file via `harness key set`): fill any - // provider slot still empty after env + the login loaders. env always wins. + // An explicit provider or exact catalog model can observe only its + // candidate providers. Load those stored keys now; model/fallback/resume + // surfaces fill the rest on demand. Aliases/default startup still need all + // keys because availability participates in their routing tie-break. + const selective_stored_keys = hasSelectiveStoredKeyScope(model_flag); + var stored_keys_loaded = !selective_stored_keys; if (keys_cli.homeEnv(environ_map)) |home| { - for (provider_mod.provider_specs, &keys.values, &keys.sources) |spec, *value, *source| { - if (value.* == null) { - if (keys_cli.loadStoredKey(io, arena, home, spec.id)) |key| { - value.* = key; - source.* = .stored; - } + if (selective_stored_keys) { + keys_cli.loadMissingStoredKeys(io, gpa, arena, home, &keys, startupStoredKeyScope(model_flag, true)); + // Preserve the old diagnostics/fallback behavior when none of the + // selective candidates has any credential: only failed launches + // pay for the exhaustive scan. + if (keys.defaultProvider()) |_| {} else |_| { + keys_cli.loadMissingStoredKeys(io, gpa, arena, home, &keys, .all); + stored_keys_loaded = true; } - } + } else keys_cli.loadMissingStoredKeys(io, gpa, arena, home, &keys, .all); } - // Discover the account-scoped Codex catalog before model selection. This - // mirrors openai/codex: 5-minute cache keyed by installed Codex version, - // authenticated /models refresh, then stale/native/baked offline fallback. - // Model names, rollout visibility, and context windows therefore do not - // need a graff release. No Codex login/cache → the baked fallback remains. - if (keys_cli.homeEnv(environ_map)) |home| { - const codex_home = environ_map.get("CODEX_HOME") orelse - (std.fmt.allocPrint(arena, "{s}/.codex", .{home}) catch ""); - models_cache.loadCodexCatalog( - io, - gpa, - arena, - home, - codex_home, - keys.get("codex") orelse "", - keys.codex_account, - false, - ); + const home = keys_cli.homeEnv(environ_map) orelse ""; + const codex_home = environ_map.get("CODEX_HOME") orelse + (std.fmt.allocPrint(arena, "{s}/.codex", .{home}) catch ""); + var model_catalog: models_cache.LazyCodexCatalog = .{ .codex_home = codex_home }; + const saved_model: ?serde.SavedModel = if (model_flag == null) serde.loadModel(io, arena, home) else null; + // Dynamic Codex discovery is observable only when startup may select + // Codex. Explicit/saved non-Codex launches defer the version subprocess, + // native-cache parse, and possible refresh until a model surface needs it. + if (startupNeedsCodexCatalog(keys, model_flag, saved_model)) + model_catalog.ensure(io, gpa, arena, home, keys.get("codex") orelse "", keys.codex_account); + if (home.len != 0) { // Apply the independent models.dev price/context overlay after routing // discovery. Provider-specific Codex windows remain authoritative. models_cache.loadOverlay(io, arena, home); @@ -164,7 +253,7 @@ pub fn resolveKeys(io: Io, gpa: Allocator, arena: Allocator, environ_map: anytyp }; const nm = pricing.resolveModelName(keys, mname) orelse std.process.fatal("unknown --model '{s}' — run `graff models refresh` or see /models", .{mname}); default_provider = keys.providerFor(nm) catch std.process.fatal("no key/login for --model '{s}' — see /models", .{mname}); - } else if (serde.loadModel(io, arena, keys_cli.homeEnv(environ_map) orelse "")) |saved| { + } else if (saved_model) |saved| { preferred_provider = saved.pid; // No --model flag: resume the model chosen last session only if that // exact provider/model pair is still in the catalog; model names can be @@ -188,7 +277,7 @@ pub fn resolveKeys(io: Io, gpa: Allocator, arena: Allocator, environ_map: anytyp } } } - return .{ .keys = keys, .default_provider = default_provider, .stale_saved_model = stale_saved_model, .preferred_provider = preferred_provider, .codex_account = codex_account }; + return .{ .keys = keys, .default_provider = default_provider, .stale_saved_model = stale_saved_model, .preferred_provider = preferred_provider, .codex_account = codex_account, .model_catalog = model_catalog, .stored_keys_loaded = stored_keys_loaded }; } pub const SystemPrompt = struct { diff --git a/src/subagent.zig b/src/subagent.zig index 6b1ab09e..99547866 100644 --- a/src/subagent.zig +++ b/src/subagent.zig @@ -225,7 +225,7 @@ pub fn runSub(ctx: ToolCtx, kind: []const u8, label: []const u8, prompt: []const .task = util.utf8Prefix(prompt, 160), .tools = used_tools, .ok = run_ok, - .context_tokens = agent.last_context_tokens, + .context_tokens = agent.effectiveContextTokens(), }); } if (telemetry.g_telem) |t| t.runEvent(&fp, sys_override != null, run_ok, run_ms, used_tools); diff --git a/src/telemetry.zig b/src/telemetry.zig index 074e5c05..3f5289ec 100644 --- a/src/telemetry.zig +++ b/src/telemetry.zig @@ -20,6 +20,7 @@ const g_cost = &pricing.g_cost; const util = @import("util.zig"); const utf8Prefix = util.utf8Prefix; const unixMs = util.unixMs; +const http = @import("http.zig"); // For the session run id (score/run join key) and the propose-site // fingerprint check. No cycle: scoring imports only std + pricing. @@ -377,6 +378,7 @@ pub const Telemetry = struct { } pub fn postOtlp(client: *std.http.Client, url: []const u8, payload: []const u8) void { + http.waitForClientReady(client.io); _ = client.fetch(.{ .location = .{ .url = url }, .method = .POST, diff --git a/vision.md b/vision.md new file mode 100644 index 00000000..bc59c0e0 --- /dev/null +++ b/vision.md @@ -0,0 +1,45 @@ +# Vision: From Evolutionary Harness to Agentic Operating System (AOS) + +## Core Concept +Transition the `graff` architecture from a tool that executes agentic tasks into a **Self-Optimizing Orchestration Layer**. The ultimate goal is an **Agentic Operating System (AOS)** that treats intelligence, cost, and latency as dynamic resources to be orchestrated via a hierarchical, evolutionary, and recursive mesh. + +--- + +## Phase 1: The Evolutionary Substrate (Foundational) +*The current state: Building the capacity for "biological" iteration of agent logic.* +- [x] **Lineage Tracking:** Implementation of Darwinian-style trajectory logs (`harness.trajectory.jsonl`) to record the "genealogy" of every turn. +- [x] **Prompt Mutation:** Enabling changes in system prompts and agent personas as "genetic mutations." +- [x] **Fitness Ledger:** Implementation of HMAC-signed, tamper-proof scoring to allow for verifiable, automated agent improvement. +- [ ] **Automated Population Testing:** Tools to run multiple "variants" of a task to identify successful prompt/workflow configurations. + +--- + +## Phase 2: Intelligent Resource Orchestration (The Smart Router) +*Moving from "static" model selection to "predictive" economic optimization.* +- [ ] **Complexity Profiling:** Analyzing task complexity (e.g., via a tiny local QAT model) before execution to classify tasks into Tiered Intelligence Levels. +- [ ] **The Cache-Switching Predictor:** A mathematical model to manage the "Cache-Switching Paradox." The router must decide if a model switch is economically viable based on: + - **Current Context Momentum** (the weight of the existing cache). + - **Predicted Success Probability** (using evolutionary traces to know if a cheaper model is likely to fail). + - **Cost-Benefit Gradient** (balancing $ per token vs. the cost of error/retries). +- [ ] **Dynamic Escalation/De-escalation:** Automatically promoting a task to a higher-tier model if a failure is detected, or downgrading a task once stability is achieved. + +--- + +## Phase 3: Recursive Heterogeneity (The Hierarchical Mesh) +*The leap from a single "brain" to a mesh of specialized, on-the-fly agents.* +- [ ] **Recursive Routing:** Enabling every subagent to act as its own "Mini-Router," selecting the optimal model for its specific sub-task on the fly. +- [ ] **Context Distillation (Information Scaling):** + - **Downscaling (Compression):** Condensing massive high-intelligence context into a "Minimal Operational Instruction Set" (MOIS) for lightweight subagents. + - **Upscaling (Expansion):** Re-injecting low-precision subagent outputs into the high-precision root context for reasoning. +- [ ] **Intelligence Arbitrage:** Optimizing the "Success-per-Dollar" ratio by matching the precise level of intelligence required to the specific task complexity. + +--- + +## Phase 4: The Agentic Operating System (AOS) +*The ultimate vision: An autonomous, distributed execution environment.* +- [ ] **Hybrid/Distributed Execution:** seamless movement of tasks between local (QAT/Edge) and cloud (API) environments. +- [ ] **Self-Evolving Meshes:** A system that learns not just the best prompts, but the best *organizational structures* for different classes of software engineering work. +- [ ] **Autonomous Optimization:** An engine that constantly re-calculates the cost-efficiency of its own hierarchy, evolving its own "management logic" over time. + +--- +*“Don't just run an agent. Orchestrate an evolution.”*