diff --git a/CHANGELOG.md b/CHANGELOG.md index 09c68d1..cc872e8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,71 @@ type. ## [Unreleased] +## [0.5.0] - 2026-07-23 + +A large hardening + feature run: all planned tiers, the critical/high/medium audit +findings, and the async-job lifecycle over MCP. + +### Added + +- **Async job lifecycle over MCP.** `build_async`/`run_async` jobs now persist + across a server restart (a job still `running` when the process died reloads as + `interrupted`) and expose task-level progress — `done`/`total`/`phase`/`message` — + surfaced by `job_status`. Persistent by default (`~/.misterdev/jobs.json`, + `MISTERDEV_STATE_DIR` override). +- **`full_rewrite` escalation rung** — a structurally different retry (rewrite the + whole target region) between context-widen and model-swap. +- **Escalation sub-steps execute as real gated child tasks** (depth-guarded); + **proactive keystone splitting** of high-fan-in tasks; a **size/verifiability + invariant** flags over-large or unverifiable decomposed tasks. +- **Runtime FailureLog read-back** (a task sees its own prior failures within the + run) and a **schedulable, benchmark-gated evolution pass**. +- **compile_view per-language adapter registry** — real rust/typescript/go/swift/ + csharp diagnostic parsers — and **named-symbol definitions surfaced** on + "cannot find X" / type-mismatch diagnostics. +- **Documentation tool (context7/fetch) mounted by default** (isolation-gated, + opt-out via `mcp.docs_tool`). + +### Changed + +- **spec-as-tests is default-on** (advisory). The **mutation gate scores every + changed source file**, not just the largest. The **edit region is never + truncated** by the context budget. The **independent judge** detects a + same-model config as non-independent instead of silently claiming independence. + +### Fixed + +- **CRITICAL — the integration gate reverts a wave that breaks the build.** A + post-wave suite that no longer compiles/collects was waved through as unparseable + (a false-GREEN); it is now reverted while genuinely ambiguous failures are still + left alone. +- A **zero-test gate** and a **MANIFEST acceptance pass-through with no prior + objective gate** are hard-rejected. **dotnet/VSTest counts sum across all + projects**; the **C# classifier anchors on the Roslyn `error CSxxxx:` prefix**. + The decomposer **prunes dependencies pointing at trimmed tasks**; model selection + **widens instead of silently defaulting** when the top tier is empty; + `RealTimeAligner` tolerates a malformed `consensus.json`; an unchanged file is no + longer treated as fully mutated; deferral task ids containing ` - ` are preserved. + +### Security + +- **Reject an untrusted MCP `runtimeHint`** outside the npx/uvx allowlist, closing + arbitrary-binary execution via a discovered server. + +### Performance + +- **SymbolGraph per-file index** (file-scoped queries are no longer + O(all-symbols)); the **Gatekeeper git-diff is memoized** within one gate run + (3 subprocess passes → 1). + +### Removed / robustness + +- **web-verify no longer leaks a dev-server** on a silent readiness wait; + **worktrees are torn down** if batch prep raises; **venv setup is + timeout-bounded**. +- Orphaned dead modules `held_out_oracle` and `early_abort`, and the unused + `ModelSelector.is_ready`. + ### Changed - **Integration gate: identity-based regression detection on a red baseline.** The diff --git a/GAP_ANALYSIS.md b/GAP_ANALYSIS.md new file mode 100644 index 0000000..6cb228c --- /dev/null +++ b/GAP_ANALYSIS.md @@ -0,0 +1,86 @@ +# GAP_ANALYSIS — project-completer (donor) → misterdev (canonical) + +Phase 1, read-only. Verdict up front, evidence below. No code changed. + +**Scope:** project-completer is a ~10k-LOC JavaScript MCP server (`/Volumes/A/autocoder/project-completer`) covering the same domain as misterdev (autonomously audit/fix/complete/build projects). misterdev is the maintained, published product (PyPI + MCP registry, `dcondrey`). Goal: reimplement the donor's *good ideas* as idiomatic Python, not merge the JS. + +--- + +## TL;DR recommendation + +**Port #1 — Reference-guided build (`reference_dir` digest).** Highest value-per-unit-effort. misterdev already solves the hard part (multi-language symbol extraction via tree-sitter topography across 9 languages); the donor's `reference-analyzer` is a thin digest layer on top of capability misterdev *already has*. It's additive, offline-testable, self-contained, fits "one reviewable commit + tests," and is directly the workflow you're using right now (porting a reference implementation into a target). Effort **S–M**, risk **MED**. + +**Port #2 (strategic, not a quick port) — Async job lifecycle over MCP.** The donor's single most differentiated architecture: `start → run_id → status / pause / resume / stop`. misterdev's MCP `build`/`run` are **fully synchronous** — they block until the entire build finishes and return a string. For any non-trivial autonomous build over MCP, the client can't monitor, pause, or survive a timeout. Highest raw value, but effort **L** and risk **HIGH** (deep orchestrator rework, hard to test offline). Deserves its own dedicated effort, not folded into a quick port. **Critically: do NOT copy the donor's concurrency model — that's exactly where its 5 critical bugs live** (see "Bugs not to port"). misterdev's existing worktree/parallel scheduler is already race-safe; reuse it. + +Everything else the donor does, misterdev already does at equal-or-better quality. Details below. + +--- + +## Inventories (verified against source) + +### project-completer — 18 MCP tools, 3 workflows + +| Workflow | Tools | +|---|---| +| **Completion** (audit→fix→verify) | `start_completion`, `get_completion_status`, `get_completion_findings`, `get_completion_report`, `pause_completion`, `resume_completion`, `stop_completion` | +| **Evolution** (analyze→plan→approve→execute) | `analyze_project`, `create_evolution_plan`, `get_evolution_plan`, `approve_plan`, `execute_plan`, `get_execution_status` | +| **Spec build** (decompose→scaffold→implement→converge) | `build_from_spec` (supports `reference_dir` porting), `get_spec_build_status`, `stop_spec_build` | + +Key `lib/*.js`: `orchestrator`, `evolution-orchestrator`, `spec-build-orchestrator`, `scheduler` (concurrency + file-lock + budget), `model-router` (complexity→model), `convergence` (diminishing-returns detector), `strategies` (per-stack audit prompts), `knowledge` (approve/reject learning), `library-intel` (ecosystem detect + `npm/pip audit`), `reference-analyzer` (extract interfaces/data-models/module-graph from a donor impl), `sandbox` (native/Docker), `write-queue` (serialize SQLite). Persistence: `better-sqlite3`. + +### misterdev — 5 MCP tools, 8 CLI subcommands, deep subsystems + +- **MCP tools:** `list_projects`, `status`, `scan`, `build` (plan+execute a goal, synchronous), `run` (execute planned tasks, synchronous). +- **CLI:** `scan`, `list`, `status`, `report`, `run`, `build`, `plan`, `interactive` (+ `mcp`), plus natural-language routing. +- **Subsystems:** planning (`decompose_spec`, `advisor.recommend_work`, `sovereign` strategies), verification (build/test/lint/typecheck/mutation/smoke/web/held-out-oracle gates), learning (`lesson_store`, `failure_taxonomy`, `warm_start`), evolution (`EvolutionLoop`, tool invention), economics (ledger-based UCB `model_selector`), context (tree-sitter topography for 9 languages), integration (`MCPManager`). + +--- + +## Gap table + +Legend — **Have**: equivalent / partial / missing. **Value/Effort/Risk** apply only to *missing* or *partial* rows worth porting. + +| # | Donor capability | misterdev today | Have | Port value | Effort | Risk | Verdict | +|---|---|---|---|---|---|---|---| +| 1 | **Reference-guided build** (`reference-analyzer`: extract interfaces / data-models / module graph from a donor impl, feed the builder) | `decompose_spec` + tree-sitter topography (9 langs) exist, but **no reference-porting layer**; `build` takes a goal string only | **missing** (spec decomposition partial) | **HIGH** | **S–M** | MED | **PORT FIRST.** Topography already does the hard multi-language extraction; this is a digest + prompt-wiring layer. | +| 2 | **Async job lifecycle over MCP** (`start`→`run_id`→`status`/`pause`/`resume`/`stop`) | MCP `build`/`run` are **synchronous, blocking, return a string** (verified: no `run_id`/job/thread/asyncio in `mcp_server.py`) | **missing** | **HIGH** | **L** | HIGH | **PORT SECOND, standalone.** Real defect for MCP use. Big lift; do NOT copy donor concurrency (buggy). | +| 3 | **Human-in-loop plan approval over MCP** (`create_plan`→`approve_items`/`reject_items`→`execute`) | Interactive `plan` CLI + `advisor.recommend_work`; **no MCP approve/reject surface** | **partial** | MED | M | MED | Defer. Overlaps existing planning; mostly an MCP-surface exposure of what CLI already does. | +| 4 | **Structured findings query** (`get_completion_findings` by severity/file/status) | Produces build *reports* + audit trail; **no queryable findings store** | **partial** | LOW–MED | M | LOW | Defer. Requires a findings store misterdev's model doesn't currently keep. | +| 5 | **Ecosystem security audit** (`library-intel`: `npm audit` / `pip-audit` / `cargo audit`) | `detection.py:139-148` **already** resolves `cargo audit` / `npm audit --omit=dev` / `pip-audit` as an advisory gate | **equivalent** | — | — | — | **SKIP — would duplicate.** | +| 6 | **Preference learning** (`knowledge`: approve/reject/success/failure) | `lesson_store`, `failure_taxonomy`, `warm_start`, metacognition | **equivalent** | — | — | — | Skip. | +| 7 | **Model routing** (`model-router`: complexity→opus/sonnet/haiku) | `model_selector` (ledger-driven UCB, quality-per-dollar under a validation floor) | **equivalent (better)** | — | — | — | Skip — misterdev's is superior. | +| 8 | **Convergence detector** (`convergence`) | `metacognition` / `advisor` diminishing-returns handling | **equivalent** | — | — | — | Skip (donor's has div-by-zero bugs anyway). | +| 9 | **Concurrency scheduler + file lock** (`scheduler`) | Parallel isolated git worktrees + integration gate + auto-bisect revert | **equivalent (safer)** | — | — | — | Skip. | +| 10 | **Sandbox** (native/Docker) | `container_env` + `evolution/sandbox` | **equivalent** | — | — | — | Skip. | +| 11 | **Per-stack audit prompts** (`strategies`) | `guidance/*` per-language conventions + analyzer prompts | **equivalent** | — | — | — | Skip. | +| 12 | **SQLite persistence + write-queue** | YAML/project registry + `.orchestrator/` progress (content-hash replay) | **equivalent (different model)** | — | — | — | Skip — no reason to adopt SQLite. | + +**Net:** of 12 donor capability clusters, 8 are already equal-or-better in misterdev (skip), 2 are partial/low-priority (defer), and **2 are genuine high-value gaps** (rows 1 and 2). + +--- + +## Bugs not to port (from donor `todo.md`: 5 critical / 14 high) + +The donor's open criticals cluster almost entirely in its **async/concurrency machinery** — the exact subsystem behind Port #2. Reimplementing that surface on misterdev's existing race-safe worktree scheduler *avoids* these by construction; copying the JS structure would import them: + +- **C-006 / C-009** scheduler: file lock not atomic with CLI exec; budget check reads stale value under concurrent dispatch → overruns. +- **C-007 / C-008** CLI wrapper: double-resolve on exit+error race; SIGKILL timer not cleared on normal exit (leaks event loop). +- **C-010 / H-020** server: no guard against concurrent `start_completion` on same dir (DB/file corruption); `project_dir` unvalidated (path traversal). +- **CLU-003 / CLU-004** systemic: concurrent-run corruption; resource exhaustion (DB conns / orchestrators / AbortControllers never released). +- **H-021 / H-022** convergence: div-by-zero and off-by-one → premature convergence (another reason to skip row 8). +- **SYS-001** silent error swallow across `git.js` / `fix.js` / `report.js` (catch-return-empty) — violates misterdev's "no silent swallow" baseline; do not reproduce. + +--- + +## Recommended port order + +1. **Reference-guided build** — `reference_dir` digest built on existing topography; new optional param on `build` (+ optional CLI flag), offline-testable against a fixture reference tree. **← start here.** +2. **Async MCP job lifecycle** — dedicated effort; new `run_id`-based tools alongside (not replacing) the synchronous ones; reuse misterdev's worktree scheduler; never copy the donor's concurrency code. +3. *(defer)* MCP plan-approval surface (row 3). +4. *(defer)* Findings query store (row 4). + +Rows 5–12: **do not port** (duplicate existing, equal-or-better misterdev capability). + +--- + +*Phase 2 will implement exactly one capability — your pick — as a single reviewable commit with tests, `ruff` + `pytest` green, no heavy deps, public surface intact.* diff --git a/misterdev/__init__.py b/misterdev/__init__.py index 3c4af63..a0fb80a 100644 --- a/misterdev/__init__.py +++ b/misterdev/__init__.py @@ -1,3 +1,3 @@ """misterdev — an autonomous, extensible LLM build orchestrator.""" -__version__ = "0.4.1" +__version__ = "0.5.0" diff --git a/misterdev/agent.py b/misterdev/agent.py index fe52692..d693ae2 100644 --- a/misterdev/agent.py +++ b/misterdev/agent.py @@ -1,9 +1,10 @@ -import concurrent.futures +import json import re +import shlex import subprocess from datetime import datetime, timezone from pathlib import Path -from typing import Optional, Dict, Any +from typing import Any, Callable, Dict, Optional from rich.console import Console from rich.markdown import Markdown @@ -24,6 +25,7 @@ from misterdev.core.context.scratchpad import Scratchpad from misterdev.core.planning.decomposer import ( decompose_spec, + split_keystone_tasks, topological_sort, format_plan, ) @@ -49,6 +51,10 @@ from misterdev.core.reporting.report import BuildReport from misterdev.core.execution.project import Project from misterdev.core.execution.deferral import DeferralBook +from misterdev.core.execution.env_learnings import EnvLearnings +from misterdev.core.execution.parallel import ParallelExecutionMixin +from misterdev.core.execution.integration_gate import IntegrationGateMixin +from misterdev.utils.file_utils import atomic_write, orchestrator_state_file from misterdev.core.models import Task from misterdev.analyzers.project_analyzer import ( analyze_project, @@ -57,14 +63,18 @@ detect_lint_command, detect_typecheck_command, ) +from misterdev.analyzers.reference_digest import build_reference_digest from misterdev.core.planning.advisor import recommend_work +from misterdev.core.planning.plan_store import ( + approved_items, + save_plan, +) from misterdev.llm.client import BudgetExceededError from misterdev.task_executors.markdown_plan_executor import ( MarkdownPlanExecutor, ) from misterdev.agent_helpers import ( ProgressReporter, - _WorktreeProjectView, _apply_budget_ceiling, _budget_exhausted, _check_golden_config, @@ -72,6 +82,8 @@ _warn_if_baseline_broken, _warn_if_no_test_gate, _warn_if_test_gate_is_noop, + worktree_healthcheck_command, + worktree_setup_command, ) from misterdev.config import get_setting from misterdev.logging_setup import setup_logger @@ -87,13 +99,31 @@ CONVERGENCE_CEILING = 25 -class ProjectOrchestrator: +class ProjectOrchestrator(ParallelExecutionMixin, IntegrationGateMixin): """Main orchestrator with Sovereign Grounded workflow.""" def __init__(self): self.registry = ProjectRegistry() self.last_build_succeeded = True self.last_build_cost = 0.0 + # Cooperative-stop plumbing for background jobs (see core.execution.jobs). + # request_stop() lowers the active run's budget so the next model call + # trips the existing graceful-halt path; None until a run loads a client. + self._active_client: Any = None + self._stop_requested = False + + def request_stop(self) -> None: + """Cooperatively cancel the in-flight build/run, if any. + + Reuses the budget kill-switch instead of interrupting the task loop: + the active client's ceiling is dropped to 0 so its next call raises + BudgetExceededError, which build()/the pipeline already degrade to a + partial report. Safe to call before a client exists (the flag is + honored when the run loads one) and idempotent. + """ + self._stop_requested = True + if self._active_client is not None: + _apply_budget_ceiling(self._active_client, 0.0) def scan_directory(self, path: str | Path): self.registry.discover_projects(path) @@ -130,6 +160,7 @@ def run_project( tasklist: Optional[str] = None, budget: Optional[float] = None, proceed: bool = False, + progress_cb: Optional[Callable[..., None]] = None, ): """Run pending devplan tasks with dependency-aware orchestration. @@ -150,6 +181,11 @@ def run_project( # budget (this path previously had no CLI override at all). if budget is not None: _apply_budget_ceiling(project.llm_client, budget) + # Expose this run's client so a background job's request_stop() can trip + # the budget kill-switch; honor a stop that arrived before it existed. + self._active_client = project.llm_client + if self._stop_requested: + _apply_budget_ceiling(self._active_client, 0.0) _check_golden_config(project.config) if project.env_manager: project.env_manager.setup() @@ -200,6 +236,11 @@ def run_project( ) run_parallel = get_setting(project.config, "orchestrator", "run_parallel") + # The branch a completed/failed task must return to; captured once so a + # task that strands HEAD (via an unhandled error) can be recovered instead + # of later tasks piling their merges onto the dead branch. + _bb = run_git("git rev-parse --abbrev-ref HEAD", project.path) + base_branch = _bb.stdout.strip() if _bb and _bb.returncode == 0 else None completed_ids = set(progress.completed) failed_ids: set[str] = set() deferred_ids: set[str] = set() @@ -222,6 +263,17 @@ def run_project( logger.info(f"Running {len(tasks)} pending tasks for {project.name}.") reporter = ProgressReporter(len(tasks)) + + def _emit_progress(phase: str) -> None: + """Best-effort task-level progress to an async job's reporter.""" + if progress_cb is None: + return + try: + progress_cb(done=len(completed_ids), total=len(tasks), phase=phase) + except Exception as e: # a progress callback must never break the run + logger.debug(f"Progress callback failed (non-fatal): {e}") + + _emit_progress("executing") remaining = list(tasks) aborted = False wave = 0 @@ -312,9 +364,11 @@ def apply_result(task, result) -> bool: language=lang, ) changes.record_task_changes(task.id, modified) + _emit_progress(f"wave {wave}") return False failed_ids.add(task.id) progress.mark_failed(task.id) + _emit_progress(f"wave {wave}") consecutive_failures += 1 if consecutive_failures >= max_consecutive_failures: logger.error("Aborting run: too many consecutive failures.") @@ -338,6 +392,7 @@ def apply_result(task, result) -> bool: logger.warning(f"Halting run: {e}") aborted = True break + self._recover_to_base_branch(project, base_branch) for task, result, error in batch: if isinstance(error, BudgetExceededError): aborted = True @@ -360,6 +415,7 @@ def apply_result(task, result) -> bool: except Exception as e: logger.error(f"Task {task.id} raised: {e}") result = None + self._recover_to_base_branch(project, base_branch) if apply_result(task, result): aborted = True break @@ -373,6 +429,37 @@ def apply_result(task, result) -> bool: ) reporter.summary(cost=cost) + # Structured close-out (parity with the build pipeline): classify this run's + # failures/parks into the taxonomy + write run_summary.json, and record the + # durable env facts. The `run --tasks` path had neither, so a run on this + # path was undiagnosable at a glance and never fed P7's cross-run memory. + try: + import time + + by_id = {t.id: t for t in tasks} + failed_items = [ + (tid, self._task_failure_text(by_id[tid])) + for tid in failed_ids + if tid in by_id + ] + deferred_items = [ + ( + d["id"], + f"{d.get('reason', '')} {' '.join(d.get('questions', []))}".strip(), + ) + for d in deferrals + ] + self._emit_run_summary( + project, + len(completed_ids), + failed_items, + deferred_items, + time.time() - reporter.start_time, + ) + self._record_env_learnings(project) + except Exception as e: # a summary/learning write must never sink a run + logger.warning(f"Run close-out (summary/env-memory) skipped: {e}") + # Walk-away close-out: if any task parked for the user, write the question # book (preserving answers already typed) and surface a concise pointer so # a returning user knows exactly what to do to finish the build. @@ -388,6 +475,26 @@ def apply_result(task, result) -> bool: if len(deferrals) > 12: console.print(f" [dim]... and {len(deferrals) - 12} more[/]") + def _recover_to_base_branch(self, project, base_branch) -> None: + """Return HEAD to the run's base branch if a task stranded it. + + The executor creates a ``task/`` branch before running; on a clean + success or failure it merges/reverts back to base, but an UNHANDLED + exception can bypass that cleanup and leave HEAD on the dead task branch — + after which every later sequential task piles its merge onto that branch + instead of main. Called after each task, this is a no-op when HEAD is + already on base (the normal case) and a reset-to-base otherwise.""" + if not base_branch: + return + proc = run_git("git rev-parse --abbrev-ref HEAD", project.path) + cur = proc.stdout.strip() if proc and proc.returncode == 0 else "" + if cur and cur != base_branch: + run_git("git reset --hard", project.path) + run_git(f"git checkout {shlex.quote(base_branch)}", project.path) + logger.warning( + f"Recovered HEAD from stranded branch '{cur}' back to '{base_branch}'." + ) + def _requirements_preflight(self, project, tasks, proceed: bool) -> bool: """Review the plan for user-supplied inputs up front. Writes REQUIREMENTS.md, injects any typed decision answers, prints a summary, and — the smart gate — @@ -417,6 +524,7 @@ def _requirements_preflight(self, project, tasks, proceed: bool) -> bool: for key, answer in book.load_answers().items(): for r in reqs: if r["key"] == key: + r["answered"] = True for t in tasks: if t.id in r.get("task_ids", []): t.processor_data["user_answer"] = answer @@ -537,7 +645,13 @@ def run_task(self, project_path: str | Path, task_id: str): logger.error(f"Task {task_id} raised: {e}") project.task_manager.update_task_status(task_id, "failed") - def build(self, project_path: str | Path, args: str = "") -> str: + def build( + self, + project_path: str | Path, + args: str = "", + reference_dir: str | None = None, + progress_cb: Optional[Callable[..., None]] = None, + ) -> str: project = self._get_or_register(project_path) if not project: return "Error: could not load project" @@ -547,6 +661,20 @@ def build(self, project_path: str | Path, args: str = "") -> str: prompt = " ".join(remaining) mode = resolve_mode(prompt, project.path) + # Optional reference implementation to port from. Validate the path up + # front (fail fast, like the dirty-tree check) rather than planning + # against nothing; the digest itself is read-only and offline. + reference_digest = "" + if reference_dir: + try: + reference_digest = build_reference_digest( + reference_dir, + cache_dir=project.path / ".orchestrator", + ) + except (ValueError, OSError) as e: + logger.error(f"Reference analysis failed: {e}") + return f"Error: reference dir unusable ({e})." + logger.info(f"Build started: mode={mode.value}, flags={flags}") start_time = datetime.now(timezone.utc) @@ -566,6 +694,13 @@ def build(self, project_path: str | Path, args: str = "") -> str: # Propagate budget to LLM client _apply_budget_ceiling(project.llm_client, flags.budget) + # Expose this run's client so a background job's request_stop() can trip + # the budget kill-switch. Honor a stop that arrived before the client + # existed (dropped budget halts the very first call). + self._active_client = project.llm_client + if self._stop_requested: + _apply_budget_ceiling(self._active_client, 0.0) + # Preflight: fail fast on a retired/misrouted model id before spending # the analysis phase on calls that would 404 mid-run. if not flags.dry_run: @@ -594,7 +729,15 @@ def build(self, project_path: str | Path, args: str = "") -> str: _warn_if_test_gate_is_noop(assessment, report) return self._run_pipeline( - project, prompt, mode, flags, assessment, env_activate, report + project, + prompt, + mode, + flags, + assessment, + env_activate, + report, + reference_digest=reference_digest, + progress_cb=progress_cb, ) except BudgetExceededError as e: return self._halt_on_budget(project, report, e) @@ -650,6 +793,154 @@ def _persist_learning(self, project: Project, report: BuildReport) -> None: except Exception as e: logger.warning(f"Solved-task indexing failed (non-fatal): {e}") report.degraded_subsystems.append(f"Solved-task indexing: {e}") + try: + self._record_env_learnings(project) + except Exception as e: + logger.warning(f"Env-memory persist failed (non-fatal): {e}") + report.degraded_subsystems.append(f"Env-memory persist: {e}") + try: + self._write_run_summary(project, report) + except Exception as e: + logger.warning(f"Run summary write failed (non-fatal): {e}") + report.degraded_subsystems.append(f"Run summary: {e}") + + @staticmethod + def _task_failure_text(task) -> str: + """The best-available failure text for a terminal non-success task: the + error stashed on the error path, else the last execution result's message + and logs. '' when nothing is recorded.""" + stored = (getattr(task, "processor_data", None) or {}).get("failure_text") + if stored: + return str(stored) + hist = getattr(task, "execution_history", None) or [] + if hist: + last = hist[-1] + return f"{getattr(last, 'message', '') or ''}\n{getattr(last, 'logs', '') or ''}" + return "" + + def _write_run_summary(self, project: Project, report: BuildReport) -> None: + """Classify the build pipeline's failures and write the one-glance summary + (feeds P7/P10; answers "why did this run underperform" at a glance).""" + end = report.end_time or datetime.now(timezone.utc) + elapsed = (end - report.start_time).total_seconds() + failed_items = [(t.id, self._task_failure_text(t)) for t in report.failed_tasks] + deferred_items = [ + (t.id, self._task_failure_text(t)) for t in report.deferred_tasks + ] + self._emit_run_summary( + project, len(report.completed_tasks), failed_items, deferred_items, elapsed + ) + + def _emit_run_summary( + self, + project: Project, + completed: int, + failed_items: list, + deferred_items: list, + elapsed_seconds: float, + ) -> None: + """Classify (id, text) failure/deferral pairs into the taxonomy and write a + one-glance summary to the console and ``.orchestrator/run_summary.json``. + Shared by the build pipeline and the ``run --tasks`` path so both emit the + same signal. Never raises — a summary must not sink a finished run.""" + from misterdev.core.execution.failure_taxonomy import build_run_summary + + summary = build_run_summary( + completed, failed_items, deferred_items, elapsed_seconds + ) + atomic_write( + orchestrator_state_file(project.path, "run_summary.json"), + json.dumps(summary, indent=2), + ) + # Concise console line — the whole point is one-glance readability. + mins, secs = divmod(int(summary["elapsed_seconds"]), 60) + parts = [f"[green]{summary['completed']} done[/]"] + if summary["deferred"]: + parts.append(f"[yellow]{summary['deferred']} deferred[/]") + if summary["failed"]: + parts.append(f"[red]{summary['failed']} failed[/]") + console.print( + "[bold]Run summary[/] · " + + " · ".join(parts) + + f" · [dim]{mins}m {secs}s[/]" + ) + if summary["failure_breakdown"]: + brk = ", ".join( + f"{cat} {n}" for cat, n in summary["failure_breakdown"].items() + ) + console.print(f" [dim]failures:[/] {brk}") + top = summary["top_obstacle"] + if top: + ex = summary["exemplars"].get(top, "") + console.print( + f" [dim]top obstacle:[/] {top}" + + (f" [dim]— {ex[:120]}[/]" if ex else "") + ) + + def _record_env_learnings(self, project: Project) -> None: + """Record this run's durable environment facts for the next run. + + Loads the existing ledger and refreshes: the effective worktree + setup/healthcheck commands (resolved from the current config), and a + learned max_workers — persisted ONLY when the adaptive loop settled BELOW + the configured base (a real, contention-driven reduction). A run that held + or recovered to full concurrency clears any stale reduction, so a single + bad run never pins the project low forever. + """ + learnings = EnvLearnings.load(project.path) + setup = worktree_setup_command(project.config, project.path) + if setup: + learnings.worktree_setup_command = setup + health = worktree_healthcheck_command(project.config, project.path) + if health: + learnings.worktree_healthcheck_command = health + settled = getattr(project, "env_settled_workers", None) + base = getattr(project, "env_base_workers", None) + if settled is not None and base is not None: + learnings.max_workers = settled if settled < base else None + learnings.save(project.path) + + def propose_plan(self, project_path: str | Path, args: str = "") -> Dict[str, Any]: + """Analyze the project and persist ranked work proposals for approval. + + The non-interactive counterpart to interactive_plan(): runs analysis and + the advisor, writes the proposals to .orchestrator/proposed_plan.json + (each unapproved), and returns them so a client can review and approve a + subset before any code is edited. Spends LLM budget; edits no code. + """ + project = self._get_or_register(project_path) + if not project: + return {"error": "could not load project"} + _, flags = parse_flags(args.split() if args else []) + _apply_budget_ceiling(project.llm_client, flags.budget) + self._active_client = project.llm_client + try: + env_activate = self._setup_env(project) + assessment = self._analyze(project, env_activate) + recs = recommend_work(assessment, project.llm_client) + except BudgetExceededError as e: + return {"error": f"budget exhausted during analysis: {e}"} + return {"items": save_plan(project.path, recs)} + + def execute_plan(self, project_path: str | Path, args: str = "") -> str: + """Build the approved items from a previously proposed plan. + + Composes a single goal from the approved proposals and runs the normal + build pipeline (decompose -> execute -> verify) over it. Returns a + message when nothing is approved. Any build flags in ``args`` (budget, + parallel, ...) are preserved. + """ + project = self._get_or_register(project_path) + if not project: + return "Error: could not load project" + approved = approved_items(project.path) + if not approved: + return "No approved plan items to execute. Approve items first." + goal = "complete the following approved work: " + "; ".join( + it["title"] for it in approved + ) + build_args = f"{goal} {args}".strip() + return self.build(project_path, build_args) def interactive_plan(self, project_path: str | Path, args: str = "") -> str: """Analyze the project, recommend work, and compose a plan with the user. @@ -779,6 +1070,170 @@ def _working_tree_dirty(self, project: Project) -> str: return "" return f"{len(lines)} file(s), e.g. {lines[0][3:].strip()}" + def run_doctor(self, project_path: str | Path) -> Dict[str, Any]: + """Preflight a project for an unattended run: gather the environment facts + and route them through the pure ``doctor`` checks. Returns the aggregate + (counts + exit_code) with the per-check results under ``checks``. Never + raises — a gathering error degrades that one check, it does not crash.""" + from misterdev.core.execution import doctor as dr + + project = self._get_or_register(project_path) + if not project: + err = dr.Check( + "load project", dr.FAIL, "could not load the project", "check the path" + ) + out = dr.aggregate([err]) + out["checks"] = [err] + return out + + checks: list = [] + is_git = (Path(project.path) / ".git").exists() + checks.append(dr.check_git_repo(is_git)) + base = self._doctor_base_branch(project) + if is_git: + checks.append(dr.check_clean_tree(self._working_tree_dirty(project))) + checks.append( + dr.check_on_base_branch(self._doctor_current_branch(project), base) + ) + checks.append( + dr.check_leftover_task_branches(self._doctor_task_branches(project)) + ) + checks.append(dr.check_dangling_worktrees(self._doctor_worktrees(project))) + try: + ok, detail = project.llm_client.health_check() + except Exception as e: # a gathering error is not a model verdict + ok, detail = False, str(e) + checks.append(dr.check_models(ok, detail)) + if is_git: + checks.append(self._doctor_worktree_probe(project)) + checks.append( + dr.check_requirements(self._doctor_unsatisfied_requirements(project)) + ) + + out = dr.aggregate(checks) + out["checks"] = checks + return out + + @staticmethod + def _doctor_base_branch(project) -> str: + """The repo's base branch: ``main`` or ``master`` if present, else HEAD.""" + for name in ("main", "master"): + proc = run_git( + f"git rev-parse --verify --quiet {shlex.quote(name)}", project.path + ) + if proc is not None and proc.returncode == 0 and proc.stdout.strip(): + return name + return ProjectOrchestrator._doctor_current_branch(project) or "main" + + @staticmethod + def _doctor_current_branch(project) -> Optional[str]: + proc = run_git("git rev-parse --abbrev-ref HEAD", project.path) + if proc is None or proc.returncode != 0: + return None + return proc.stdout.strip() or None + + @staticmethod + def _doctor_task_branches(project) -> list: + proc = run_git("git branch --list task/*", project.path) + if proc is None or proc.returncode != 0: + return [] + return [ln.strip(" *").strip() for ln in proc.stdout.splitlines() if ln.strip()] + + @staticmethod + def _doctor_worktrees(project) -> list: + """Registered worktrees under the orchestrator's worktree dir (excludes the + main checkout), i.e. leftovers a prior run did not clean up.""" + proc = run_git("git worktree list --porcelain", project.path) + if proc is None or proc.returncode != 0: + return [] + found = [] + marker = str(Path(project.path) / ".orchestrator" / "worktrees") + for ln in proc.stdout.splitlines(): + if ln.startswith("worktree ") and marker in ln: + found.append(ln[len("worktree ") :].strip()) + return found + + def _doctor_worktree_probe(self, project): + """Create a throwaway worktree, prime deps + run the healthcheck, remove it. + Best-effort — any failure becomes a WARN, never crashes the doctor.""" + from misterdev.core.execution import doctor as dr + + setup_cmd = self._worktree_setup_command(project) + health_cmd = self._worktree_healthcheck_command(project) + if not setup_cmd and not health_cmd: + return dr.check_worktree_prime(None, None) + import uuid + from misterdev.tools.command import CommandTool + from misterdev.tools.git_tool import GitTool + + git = GitTool({}) + wt_root = Path(project.path) / ".orchestrator" / "worktrees" + wt_root.mkdir(parents=True, exist_ok=True) + git.worktree_prune(project) + run_id = uuid.uuid4().hex[:6] + branch = f"doctor/{run_id}" + wt_path = wt_root / f"doctor-{run_id}" + ok, out = git.worktree_add(project, str(wt_path), branch, new_branch=True) + if not ok: + return dr.Check( + "worktree prime + healthcheck", + dr.WARN, + f"could not create a throwaway worktree: {out[-120:]}", + "check git worktree support and disk space", + ) + timeout = get_setting(project.config, "orchestrator", "worktree_setup_timeout") + cmd = CommandTool({}) + prime_ok = None + health_ok = None + detail = "" + try: + if setup_cmd: + prime_ok, pout = cmd.execute( + project, setup_cmd, cwd=str(wt_path), timeout=timeout + ) + if not prime_ok: + detail = pout[-160:] + if health_cmd: + health_ok, hout = cmd.execute( + project, health_cmd, cwd=str(wt_path), timeout=timeout + ) + if not health_ok: + detail = hout[-160:] + finally: + git.worktree_remove(project, str(wt_path)) + git.branch_delete(project, branch) + return dr.check_worktree_prime(prime_ok, health_ok, detail) + + def _doctor_unsatisfied_requirements(self, project) -> list: + """Keys in .orchestrator/REQUIREMENTS.md marked MISSING with no typed answer. + Empty when the file is absent (nothing reviewed yet) or all are provided.""" + md = Path(project.path) / ".orchestrator" / "REQUIREMENTS.md" + if not md.exists(): + return [] + try: + from misterdev.core.planning.requirements import RequirementsBook + + answers = RequirementsBook( + Path(project.path) / ".orchestrator" + ).load_answers() + text = md.read_text(encoding="utf-8") + except OSError: + return [] + unsatisfied = [] + key = "" + missing = False + for line in text.splitlines(): + if line.startswith("## "): + if key and missing and key not in answers: + unsatisfied.append(key) + key = line[3:].split("—", 1)[0].strip().strip("`") + missing = False + elif line.strip().lower().startswith("- status:"): + missing = "missing" in line.lower() + if key and missing and key not in answers: + unsatisfied.append(key) + return unsatisfied + def _run_pipeline( self, project: Project, @@ -789,6 +1244,8 @@ def _run_pipeline( env_activate: Optional[str], report: BuildReport, confirm_plan: bool = False, + reference_digest: str = "", + progress_cb: Optional[Callable[..., None]] = None, ) -> str: """Phases 1.5-6: probes, spec, decompose, (confirm), execute, validate. @@ -797,6 +1254,17 @@ def _run_pipeline( task executes. """ _check_golden_config(project.config) + # Cross-run env memory: pre-tune this run from durable facts learned in + # prior runs (effective setup/healthcheck command, a backed-off max_workers) + # WITHOUT overriding any explicit project.yaml value. Best-effort: a missing + # or unreadable ledger just means no pre-tuning. Applied here, before any + # gate/worktree code reads the config. + try: + applied = EnvLearnings.load(project.path).apply_to_config(project.config) + for a in applied: + logger.info(f"Env-memory: pre-tuned {a} from a prior run") + except Exception as e: + logger.debug(f"Env-memory pre-tune skipped (non-fatal): {e}") # Make the analysis baseline failure count available to the per-task test # gate so a RED baseline doesn't reject every task: the gate then accepts a # task that leaves the suite no worse, letting a multi-failure project be @@ -860,6 +1328,12 @@ def _run_pipeline( mode, prompt, assessment, project, facts=verified_facts ) + # Prepend the reference-implementation digest (when porting from one) so + # decomposition and every task see the reference's real module/symbol map + # alongside the spec. Read-only and offline; empty when no --reference. + if reference_digest: + spec = f"{reference_digest}\n\n{spec}" + # Sovereign enhancements (metacognition, AB-MCTS) are best-effort: they # refine the spec but must not crash the build, so each degrades to the # current spec on failure rather than aborting before any work is done. @@ -924,6 +1398,9 @@ def _run_pipeline( targets=targets, staging_hint=self._staging_hint(project), ) + # Proactively split a high-fan-in keystone before ordering, so its whole + # fan-out is not blocked on one all-or-nothing attempt. + tasks = split_keystone_tasks(tasks) tasks = topological_sort(tasks) if flags.dry_run: @@ -958,7 +1435,7 @@ def _run_pipeline( tasks_this_iter = len(tasks) # Phase 4: Execution - self._execute_tasks(tasks, project, flags, report) + self._execute_tasks(tasks, project, flags, report, progress_cb=progress_cb) # Phase 5: Gates if flags.no_verify: @@ -1253,338 +1730,50 @@ def _build_fix_spec( return "\n".join(parts) @staticmethod - def _wave_commits(executor, project, tasks) -> list: - """Collect ``(task_id, sha)`` for each task that has a recorded commit, - skipping tasks with none. Shared by the regression-revert and - integration-gate paths.""" - commits = [] - for t in tasks: - sha = executor.find_task_commit(project, t.id) - if sha: - commits.append((t.id, sha)) - return commits - - def _maybe_rollback_regression( - self, - project: Project, - report: BuildReport, - assessment: ProjectAssessment, - flags: BuildFlags, - ) -> None: - """If the post-build gate failed, bisect task commits and revert the culprit.""" - if flags.no_rollback or not report.completed_tasks: - return - test_cmd = assessment.structure.test_command - if not test_cmd: - return - ex = MarkdownPlanExecutor() - if not ex._is_git_repo(project): - return - commits = self._wave_commits(ex, project, report.completed_tasks) - if not commits: - return - logger.warning("Post-build regression detected; bisecting task commits...") - culprit = ex.bisect_regression(project, commits, test_cmd) - if not culprit: - logger.info("Bisect did not isolate a single task commit.") - return - sha = dict(commits)[culprit] - if ex.revert_task_commit(project, sha): - logger.warning( - f"Regression bisected to {culprit}; commit {sha[:8]} reverted." - ) - report.key_decisions.append( - f"Regression from {culprit} auto-reverted (bisect)" - ) - - def _suite_failures( - self, - project: Project, - executor: MarkdownPlanExecutor, - test_cmd: str, - timeout: int, - cwd=None, - ) -> Optional[int]: - """Full-suite failure count: 0 when green, the parsed count when red, or - None when the count can't be parsed (caller then can't count-compare). - - ``cwd`` runs the command in a sub-project (target) directory; defaults to - the repo root.""" - from misterdev.core.verification.validator import ( - _parse_test_counts, - ) - - ok, output = executor._run_command(project, test_cmd, timeout=timeout, cwd=cwd) - if ok: - return 0 - total, failures = _parse_test_counts(output) - return failures if total > 0 else None - - @staticmethod - def _failing_ids_from_output(output: str, project: Project) -> Optional[set]: - """The SET of failing test identifiers parsed from runner output, or None - when none can be parsed (caller falls back to the count). - - Identity beats a bare count: it lets the integration gate revert a wave - that offsets a genuine fix against a NEW break (net-zero count, which - count mode waves through) and stays correct if a fix renames/reorders - tests. Reuses the FailureView parsers already validated per runner.""" - from misterdev.core.execution.failure_view import extract_failures - - lang = ( - (project.config.get("language") or "") - if getattr(project, "config", None) - else "" - ) - ids = { - f.test - for f in extract_failures(output, language=lang) - if getattr(f, "test", "") - } - return ids or None - - def _suite_failing_ids( - self, - project: Project, - executor: MarkdownPlanExecutor, - test_cmd: str, - timeout: int, - cwd=None, - ) -> Optional[set]: - """Full-suite failing-test id SET: empty when green, the parsed ids when - red, or None when unparseable (caller falls back to the count).""" - ok, output = executor._run_command(project, test_cmd, timeout=timeout, cwd=cwd) - if ok: - return set() - return self._failing_ids_from_output(output, project) - - def _integration_gate_count( - self, - project: Project, - executor: MarkdownPlanExecutor, - test_cmd: str, - wave_tasks: list[Task], - timeout: int, - baseline_failures: int, - ) -> list[str]: - """Count-mode gate for a RED baseline: revert wave commits (newest first) - only when the wave RAISED the full-suite failure count above the baseline. - - This closes the gap where, with the binary gate disabled by a red - baseline, a task gated on its own scoped tests could worsen the overall - suite and still commit. An unparseable post-wave count is left alone (we - do not revert on a number we can't read). - """ - after = self._suite_failures(project, executor, test_cmd, timeout) - if after is None or after <= baseline_failures: - return [] - logger.warning( - f"Integration gate (count): failures rose {baseline_failures} -> " - f"{after}; reverting wave commits until restored." - ) - commits = self._wave_commits(executor, project, wave_tasks) - reverted: list[str] = [] - for tid, sha in reversed(commits): - if executor.revert_task_commit(project, sha): - reverted.append(tid) - now = self._suite_failures(project, executor, test_cmd, timeout) - if now is not None and now <= baseline_failures: - break - return reverted - - def _integration_gate_ids( - self, - project: Project, - executor: MarkdownPlanExecutor, - test_cmd: str, - wave_tasks: list[Task], - timeout: int, - baseline_ids: set, - ) -> list[str]: - """Identity-mode gate for a RED baseline: revert the wave iff it introduced - a NEW failing test (one not failing at baseline), regardless of the failure - COUNT. - - Stricter and more correct than count mode: a wave that fixes test A but - breaks test B nets zero on the count and slips past ``_integration_gate_count``, - yet it introduced a real regression (B) — identity mode reverts it. A wave - that resolves none of the baseline failures and adds none (a no-op "fix" - that still merged) is surfaced as no-progress rather than silently blessed. - """ - after = self._suite_failing_ids(project, executor, test_cmd, timeout) - if after is None: - return [] # unparseable post-wave count of ids; don't revert blind - new_failures = after - baseline_ids - if not new_failures: - if baseline_ids - after: - logger.info( - "Integration gate (identity): resolved " - f"{len(baseline_ids - after)} baseline failure(s), no regressions." - ) - else: - logger.info( - "Integration gate (identity): wave added no new failures but " - "resolved none either — no progress on the failing suite." - ) - return [] - logger.warning( - f"Integration gate (identity): {len(new_failures)} new failing test(s) " - f"(e.g. {sorted(new_failures)[:2]}); reverting wave commits until restored." - ) - commits = self._wave_commits(executor, project, wave_tasks) - reverted: list[str] = [] - for tid, sha in reversed(commits): - if executor.revert_task_commit(project, sha): - reverted.append(tid) - now = self._suite_failing_ids(project, executor, test_cmd, timeout) - if now is not None and not (now - baseline_ids): - break - return reverted - - @staticmethod - def _target_regressed(after: Optional[int], baseline: Optional[int]) -> bool: - """Did a target's gate regress vs its baseline? - - ``after``/``baseline`` are :meth:`_suite_failures` results (0 green, N - count, None unparseable). A green-now gate never regressed. With no - countable baseline we can't compare, so we don't revert. A binary failure - now (None) is a regression only if the target was green (baseline 0); - otherwise compare counts. + def _wave_infra_count(results: list) -> int: + """How many of a wave's tasks FAILED on an ENVIRONMENT fault (not code). + + Scans each unsuccessful task's error/logs for an infra signature (timeout, + locked store, OOM, ...). A completed task never counts — a transient fault + it self-healed past is not contention worth backing off for. Only an + UN-recovered infra failure, the exact signal that concurrency is too high, + is counted. """ - if after == 0: - return False - if baseline is None: - return False - if after is None: - return baseline == 0 - return after > baseline - - def _integration_gate_targets( - self, - project: Project, - executor: MarkdownPlanExecutor, - targets: list[dict], - wave_tasks: list[Task], - timeout: int, - target_baselines: dict, - ) -> list[str]: - """Per-target integration gate: validate each sub-project the wave touched - with ITS own toolchain (in ITS directory), reverting only the wave commits - belonging to a target that regressed. This is the multi-target analogue of - :meth:`_integration_gate` — the last place a polyglot run would otherwise - gate with the wrong toolchain. - """ - from misterdev.core.planning.targets import select_target + from misterdev.core.execution.infra import infra_failure - reverted: list[str] = [] - for tgt in targets: - gate_cmd = tgt.get("test_command") or tgt.get("build_command") - if not gate_cmd: - continue - tname = tgt.get("name") or tgt.get("path") - tp = (tgt.get("path") or "").strip("/") - run_dir = project.path / tp if tp else project.path - owned = [ - t - for t in wave_tasks - if ( - select_target( - targets, list(t.files_to_modify) + list(t.files_to_create) - ) - or {} - ).get("path") - == tgt.get("path") - ] - if not owned: - continue - baseline = target_baselines.get(tname) - after = self._suite_failures( - project, executor, gate_cmd, timeout, cwd=run_dir - ) - if not self._target_regressed(after, baseline): + count = 0 + for _task, result, error in results: + if result is not None and getattr(result, "status", None) == "completed": continue - logger.warning( - f"Integration gate [{tname}]: regressed (baseline={baseline}, " - f"after={after}); reverting this target's wave commits." - ) - commits = [(t.id, executor.find_task_commit(project, t.id)) for t in owned] - commits = [(tid, sha) for tid, sha in commits if sha] - for tid, sha in reversed(commits): - if executor.revert_task_commit(project, sha): - reverted.append(tid) - now = self._suite_failures( - project, executor, gate_cmd, timeout, cwd=run_dir - ) - if not self._target_regressed(now, baseline): - break - return reverted - - def _integration_gate( - self, - project: Project, - executor: MarkdownPlanExecutor, - test_cmd: str, - wave_tasks: list[Task], - timeout: int, - baseline_failures: int = 0, - ) -> list[str]: - """Run the full suite after a wave; revert task commits that regressed it. - - Returns the task_ids whose commits were reverted (empty if the suite - still passes). Bisects to the single culprit when possible; if that - can't isolate it or the tree is still red afterward, reverts the - remaining wave commits (newest first) to restore a green baseline. On a - RED baseline it prefers IDENTITY mode (revert a wave that adds any new - failing test, so an offsetting fix/break can't slip through) when the - baseline's failing set was parseable, falling back to COUNT mode (revert - only a wave that raises the failure count) otherwise. + text = "" + if error is not None: + text += str(error) + if result is not None: + text += " " + str(getattr(result, "logs", "") or "") + text += " " + str(getattr(result, "message", "") or "") + if infra_failure(text): + count += 1 + return count + + def _apply_wave_tuning(self, project: Project, tuning, base: dict) -> None: + """Apply a wave's tuning by scaling the config the deep gate paths read. + + max_workers and the gate/setup timeouts are resolved via ``get_setting`` + throughout the executor and worktree code, so the one central way to make + an adapted value reach all of them is to set it on the config for the + wave. Always computed from the captured ``base`` (never the last wave's + already-scaled value) so repeated application cannot drift. Safe between + waves: the wave loop is serial here, and each wave's workers read the value + once before the parallel section starts. """ - # Prefer identity mode (revert on any NEW failing test) over count mode - # (revert only when the count rises) whenever the baseline's failing set - # was parseable: it also catches an offsetting fix/break that count mode - # nets to zero. Count mode remains the fallback for unparseable output. - baseline_ids = getattr(project, "baseline_test_failing_ids", None) - if baseline_ids: - return self._integration_gate_ids( - project, executor, test_cmd, wave_tasks, timeout, baseline_ids - ) - if baseline_failures > 0: - return self._integration_gate_count( - project, executor, test_cmd, wave_tasks, timeout, baseline_failures - ) - ok, _ = executor._run_command(project, test_cmd, timeout=timeout) - if ok: - return [] - - commits = self._wave_commits(executor, project, wave_tasks) - if not commits: - logger.warning( - "Integration gate: suite regressed but no task commits found to revert." - ) - return [] - - logger.warning("Integration gate: suite regressed; isolating culprit...") - reverted: list[str] = [] - culprit = executor.bisect_regression( - project, commits, test_cmd, timeout=timeout + orch = project.config.setdefault("orchestrator", {}) + orch["max_workers"] = tuning.max_workers + orch["worktree_setup_timeout"] = int( + round(base["setup"] * tuning.timeout_factor) ) - if culprit: - sha = dict(commits)[culprit] - if executor.revert_task_commit(project, sha): - reverted.append(culprit) - ok, _ = executor._run_command(project, test_cmd, timeout=timeout) - if ok: - return reverted - - for tid, sha in reversed(commits): - if tid in reverted: - continue - if executor.revert_task_commit(project, sha): - reverted.append(tid) - ok, _ = executor._run_command(project, test_cmd, timeout=timeout) - if ok: - break - return reverted + build = project.config.setdefault("build", {}) + build["build_timeout"] = int(round(base["build"] * tuning.timeout_factor)) + build["test_timeout"] = int(round(base["test"] * tuning.timeout_factor)) def _execute_tasks( self, @@ -1592,6 +1781,7 @@ def _execute_tasks( project: Project, flags: BuildFlags, report: BuildReport, + progress_cb: Optional[Callable[..., None]] = None, ) -> None: scratchpad = Scratchpad() aligner = RealTimeAligner(project.path) @@ -1608,6 +1798,38 @@ def _execute_tasks( completed_ids = set(progress.completed) failed_ids: set[str] = set() + + def _emit_progress(phase: str) -> None: + """Best-effort task-level progress to an async job's reporter.""" + if progress_cb is None: + return + try: + progress_cb(done=len(completed_ids), total=len(tasks), phase=phase) + except Exception as e: # a progress callback must never break the build + logger.debug(f"Progress callback failed (non-fatal): {e}") + + _emit_progress("executing") + # Skip a ready task before spawning a worktree when it is already + # satisfied (content hash unchanged since a recorded completion). Default + # on; false forces every ready task to re-run. + skip_satisfied = get_setting( + project.config, "orchestrator", "skip_satisfied_tasks" + ) + # Adaptive concurrency/timeout backoff: capture the CONFIGURED values as the + # recovery ceiling, start each run at full concurrency (factor 1.0), and + # re-tune between waves from each wave's infra-failure count. + from misterdev.core.execution.adaptive import WaveTuning, next_wave_tuning + + adaptive = get_setting(project.config, "orchestrator", "adaptive_concurrency") + adaptive_base = { + "workers": get_setting(project.config, "orchestrator", "max_workers"), + "setup": get_setting( + project.config, "orchestrator", "worktree_setup_timeout" + ), + "build": get_setting(project.config, "build", "build_timeout"), + "test": get_setting(project.config, "build", "test_timeout"), + } + wave_tuning = WaveTuning(int(adaptive_base["workers"]), 1.0) consecutive_failures = 0 aborted = False max_cost_per_task = get_setting( @@ -1724,10 +1946,14 @@ def _execute_tasks( still_waiting = [] for task in remaining: # Skip only if this exact task (id AND content hash) already - # completed. Hash-aware so a freshly decomposed plan that reuses - # generic ids (T-001...) from a prior build is not wrongly - # skipped against stale progress state. - if not progress.needs_rerun( + # completed — mark it done WITHOUT spawning a worktree, so an + # already-satisfied task doesn't pay a prime/install just to be + # re-recognized. Hash-aware so a freshly decomposed plan that + # reuses generic ids (T-001...) from a prior build is not wrongly + # skipped against stale progress state. Gated by + # skip_satisfied_tasks (default on); no gate is run on the base + # branch for this — it rests on the content hash plus the ledger. + if skip_satisfied and not progress.needs_rerun( task.id, compute_task_hash(task, project.path) ): report.completed_tasks.append(task) @@ -1789,6 +2015,12 @@ def _execute_tasks( remaining = still_waiting continue + # Apply this wave's adapted concurrency/timeouts (a no-op at full + # tuning) before dispatch, so the worktree/gate code reads the backed- + # off values under contention. + if adaptive: + self._apply_wave_tuning(project, wave_tuning, adaptive_base) + # Execute: parallel or sequential wave_completed: list[Task] = [] if flags.parallel and len(ready) > 1: @@ -1849,6 +2081,11 @@ def _execute_tasks( continue if error: logger.error(f"Task {task.id} raised: {error}") + # Preserve the failure text (merge conflict, worktree add, a + # raised exception) so the end-of-run taxonomy can classify it; + # this path records no ExecutionResult to read it back from. + if isinstance(getattr(task, "processor_data", None), dict): + task.processor_data["failure_text"] = str(error) failed_ids.add(task.id) progress.mark_failed(task.id) report.failed_tasks.append(task) @@ -1874,6 +2111,16 @@ def _execute_tasks( changes.record_task_changes(task.id, modified) if task.complexity == "architectural": aligner.certify_decision(task.title, task.description) + elif result.status == "deferred": + # Parked (walk-away input needed, or escalated to decomposition) + # — NOT a code failure, so it must not trip the consecutive- + # failure abort or be recorded as terminally failed (a re-run + # retries it). It still blocks dependents and feeds the + # convergence loop's re-decomposition, like a non-success. + task.execution_history.append(result) + failed_ids.add(task.id) + report.deferred_tasks.append(task) + consecutive_failures = 0 else: task.execution_history.append(result) failed_ids.add(task.id) @@ -1881,6 +2128,7 @@ def _execute_tasks( report.failed_tasks.append(task) consecutive_failures += 1 + _emit_progress("building") if consecutive_failures >= max_consecutive_failures: aborted = True break @@ -1924,8 +2172,46 @@ def _execute_tasks( if reverted and consecutive_failures >= max_consecutive_failures: aborted = True + # Re-tune concurrency/timeouts for the NEXT wave from THIS wave's infra + # faults: back off under contention, recover gradually when clean. + if adaptive: + infra_count = self._wave_infra_count(results) + nxt = next_wave_tuning( + infra_count, + wave_tuning, + base_workers=int(adaptive_base["workers"]), + threshold=get_setting( + project.config, "orchestrator", "adaptive_infra_threshold" + ), + timeout_factor=get_setting( + project.config, "orchestrator", "adaptive_timeout_factor" + ), + max_timeout_factor=get_setting( + project.config, "orchestrator", "adaptive_max_timeout_factor" + ), + ) + if nxt != wave_tuning: + logger.info( + f"Adaptive tuning: {infra_count} infra fault(s) this wave; " + f"next wave workers {wave_tuning.max_workers}->" + f"{nxt.max_workers}, timeout x{wave_tuning.timeout_factor:g}" + f"->x{nxt.timeout_factor:g}." + ) + wave_tuning = nxt + remaining = still_waiting + # Restore the configured concurrency/timeouts so nothing downstream sees a + # backed-off value left over from an infra-heavy wave. Expose the value the + # run settled on (and the base it started from) so the cross-run env memory + # can persist a contention-driven reduction for the next run. + if adaptive: + project.env_settled_workers = wave_tuning.max_workers + project.env_base_workers = int(adaptive_base["workers"]) + self._apply_wave_tuning( + project, WaveTuning(int(adaptive_base["workers"]), 1.0), adaptive_base + ) + # Defer any unprocessed tasks processed_ids = ( completed_ids | failed_ids | {t.id for t in report.deferred_tasks} @@ -1934,152 +2220,6 @@ def _execute_tasks( if task.id not in processed_ids: report.deferred_tasks.append(task) - @staticmethod - def _task_file_set(task: Task) -> set: - """Declared files a task will touch (modify + create). - - Tolerates non-list values (e.g. unconfigured mocks): only real lists - contribute paths, anything else is treated as "unknown / no claim". - """ - files: set = set() - for attr in ("files_to_modify", "files_to_create"): - value = getattr(task, attr, None) - if isinstance(value, list): - files.update(str(p) for p in value) - return files - - @classmethod - def _partition_disjoint(cls, ready: list[Task]) -> tuple[list, list]: - """Split tasks into a concurrent-safe group + a serial remainder. - - A task joins the concurrent group only if its declared file set is - disjoint from every task already in that group; otherwise it is - deferred to the serial remainder so overlapping writes can't interleave. - """ - concurrent_group: list = [] - serial_remainder: list = [] - claimed: set = set() - for task in ready: - files = cls._task_file_set(task) - if files & claimed: - serial_remainder.append(task) - else: - concurrent_group.append(task) - claimed |= files - return concurrent_group, serial_remainder - - def _execute_parallel( - self, ready: list[Task], executor: MarkdownPlanExecutor, project: Project - ) -> list: - """Execute a batch of independent tasks concurrently. - - In "worktree" mode each task runs in its own git worktree so parallel - edits can't collide. When the mode is left at its default and the - project is a git repo, worktree isolation is preferred automatically; - "shared" must be requested explicitly to opt out. In shared mode only - tasks with disjoint declared file sets run in the same concurrent batch; - tasks whose file sets overlap are run serially afterwards. - """ - mode = get_setting(project.config, "orchestrator", "parallel_mode") - is_git_repo = (project.path / ".git").exists() is True - # "auto" (default) isolates via worktrees on a git repo; the value itself - # carries the intent, so no fragile "was it explicitly set" detection. - prefer_worktrees = mode == "worktree" or (mode == "auto" and is_git_repo) - if prefer_worktrees and is_git_repo: - return self._execute_parallel_worktrees(ready, executor, project) - - concurrent_group, serial_remainder = self._partition_disjoint(ready) - results = [] - max_workers = get_setting(project.config, "orchestrator", "max_workers") - if concurrent_group: - with concurrent.futures.ThreadPoolExecutor( - max_workers=min(len(concurrent_group), max_workers) - ) as pool: - future_to_task = { - pool.submit( - executor.execute, task, project, use_git_branch=False - ): task - for task in concurrent_group - } - for future in concurrent.futures.as_completed(future_to_task): - task = future_to_task[future] - try: - result = future.result() - results.append((task, result, None)) - except Exception as e: - results.append((task, None, e)) - - # Tasks with overlapping file claims run one at a time. - for task in serial_remainder: - try: - result = executor.execute(task, project, use_git_branch=False) - results.append((task, result, None)) - except Exception as e: - results.append((task, None, e)) - return results - - def _execute_parallel_worktrees( - self, ready: list[Task], executor: MarkdownPlanExecutor, project: Project - ) -> list: - """Run each task in an isolated git worktree, then merge successes back. - - Worktrees are created and merged serially (git's index/worktree metadata - is not concurrency-safe); only the task bodies run in parallel. - """ - import uuid - from misterdev.tools.git_tool import GitTool - - git = GitTool({}) - wt_root = project.path / ".orchestrator" / "worktrees" - wt_root.mkdir(parents=True, exist_ok=True) - results: list = [] - prepared: list = [] - - for task in ready: - branch = f"task/{task.id}" - wt_path = wt_root / f"{task.id}-{uuid.uuid4().hex[:6]}" - ok, out = git.worktree_add(project, str(wt_path), branch, new_branch=True) - if ok: - prepared.append((task, wt_path, branch)) - else: - logger.error(f"Worktree add failed for {task.id}: {out}") - results.append( - (task, None, RuntimeError(f"worktree add failed: {out}")) - ) - - def run_one(item): - task, wt_path, branch = item - view = _WorktreeProjectView(project, wt_path) - try: - return ( - task, - executor.execute(task, view, use_git_branch=False), - None, - wt_path, - branch, - ) - except Exception as e: - return (task, None, e, wt_path, branch) - - max_workers = get_setting(project.config, "orchestrator", "max_workers") - raw = [] - if prepared: - with concurrent.futures.ThreadPoolExecutor( - max_workers=min(len(prepared), max_workers) - ) as pool: - futures = [pool.submit(run_one, item) for item in prepared] - raw = [f.result() for f in concurrent.futures.as_completed(futures)] - - for task, result, error, wt_path, branch in raw: - if result is not None and getattr(result, "status", None) == "completed": - merged, mout = git.merge_worktree(project, branch) - if not merged: - logger.error(f"Worktree merge failed for {task.id}: {mout}") - error, result = RuntimeError(f"merge failed: {mout}"), None - git.worktree_remove(project, str(wt_path)) - results.append((task, result, error)) - return results - def _interactive_prompt(self, task: Task, strategy: str = "iterative") -> str: console.print( f"\n[bold cyan]Next Task:[/] [{task.id}] {task.title} ([bold magenta]{strategy.upper()}[/])" diff --git a/misterdev/agent_helpers.py b/misterdev/agent_helpers.py index b732c5b..27f9325 100644 --- a/misterdev/agent_helpers.py +++ b/misterdev/agent_helpers.py @@ -108,6 +108,89 @@ def summary(self, cost: Optional[float] = None): ) +def worktree_setup_command(config, root) -> Optional[str]: + """The command that primes a worktree's dependencies before gating, or None. + + An explicit ``orchestrator.worktree_setup_command`` wins (``""`` disables); + otherwise it is auto-detected from the project's lockfile so a gate never pays + a full dependency install inside its own timeout. Pure (config + path in, + string out) so both the parallel worktree creation path and the per-gate + infra-reprime helper resolve the same command from one place. + """ + explicit = get_setting(config, "orchestrator", "worktree_setup_command") + if explicit is not None: + return explicit or None + if (root / "pnpm-lock.yaml").exists(): + return "pnpm install --prefer-offline" + if (root / "yarn.lock").exists(): + return "yarn install --frozen-lockfile" + if (root / "bun.lockb").exists(): + return "bun install" + if (root / "package-lock.json").exists(): + return "npm ci" + if (root / "package.json").exists(): + return "npm install --no-audit --no-fund" + return None + + +def _first_declared_dependency(root) -> Optional[str]: + """The first package name in a project's package.json dependencies (prod then + dev), or None. Used to probe that the primed node_modules actually resolves a + real dependency, not just that node runs. Best-effort: any read/parse error + yields None (the caller then skips the auto probe).""" + import json + + pkg = root / "package.json" + if not pkg.is_file(): + return None + try: + data = json.loads(pkg.read_text(encoding="utf-8")) + except (json.JSONDecodeError, OSError, ValueError): + return None + for key in ("dependencies", "devDependencies"): + deps = data.get(key) + if isinstance(deps, dict): + for name in deps: + if isinstance(name, str) and name: + return name + return None + + +def worktree_healthcheck_command(config, root) -> Optional[str]: + """A fast probe confirming a primed worktree's toolchain resolves, or None. + + An explicit ``orchestrator.worktree_healthcheck_command`` wins (``""`` disables); + otherwise it is auto-detected for node/pnpm projects — the case where a broken + or partial ``node_modules`` install silently poisons the gate. Prefers a + TypeScript toolchain resolve when the project uses TS (the dominant fresh- + worktree false-failure), else confirms the first declared dependency resolves + from the primed store. A non-node project (nothing to probe cheaply) yields + None. Pure (config + path in) so it is resolved from one place. + """ + explicit = get_setting(config, "orchestrator", "worktree_healthcheck_command") + if explicit is not None: + return explicit or None + is_node = any( + (root / f).exists() + for f in ( + "pnpm-lock.yaml", + "yarn.lock", + "bun.lockb", + "package-lock.json", + "package.json", + ) + ) + if not is_node: + return None + if (root / "tsconfig.json").exists(): + # --no-install: resolve tsc from the primed node_modules and fail fast if + # it is not there, rather than triggering a download (which would mask the + # partial-install signal we are probing for). + return "npx --no-install tsc --version" + dep = _first_declared_dependency(root) + return f"node -e \"require.resolve('{dep}')\"" if dep else None + + def _combine_commands(*cmds: Optional[str]) -> Optional[str]: """Join shell commands with ``&&`` (each parenthesised), or None if all empty. diff --git a/misterdev/analyzers/reference_digest.py b/misterdev/analyzers/reference_digest.py new file mode 100644 index 0000000..e549bdd --- /dev/null +++ b/misterdev/analyzers/reference_digest.py @@ -0,0 +1,109 @@ +"""Read-only structural digest of a reference implementation, for porting. + +The ``build`` workflow can be pointed at a *reference* implementation — an +existing project (often in a different language) whose design should be +reproduced idiomatically in the target project. This module renders a compact, +language-agnostic map of that reference's modules, public symbols, and data +models by reusing misterdev's tree-sitter symbol graph (:class:`SymbolGraph`), +so the planner is grounded in the reference's real architecture rather than a +vague prose description. + +Design constraints (a donor tool that inspired this had bugs here — we do not +repeat them): + +- **Strictly read-only.** ``SymbolGraph.build`` writes a cache to + ``/.orchestrator/topography_cache.json``; pointing it at the reference + tree would MUTATE the donor. We redirect ``cache_path`` off the reference tree + (into the target project's ``.orchestrator`` or a temp dir) so the reference + directory is never written to. +- **Bounded output.** A large reference must not blow the planning context: the + rendered map is truncated to ``max_chars`` with an explicit elision note. +- **Validated input.** The path is resolved and required to be an existing + directory; a bad path raises ``ValueError`` (the caller fails fast rather than + planning against nothing). +""" + +from __future__ import annotations + +import tempfile +from pathlib import Path + +from misterdev.core.context.topography import SymbolGraph +from misterdev.logging_setup import setup_logger + +logger = setup_logger(__name__) + +# Keep the rendered map well under a typical spec/context budget. A reference is +# a design aid, not the source of truth, so a dense-but-partial map is fine. +_DEFAULT_MAX_CHARS = 12_000 + +_HEADER = """## Reference implementation to port from: {name} + +This is a READ-ONLY structural map of an existing implementation (possibly in a +different language). Use it as a DESIGN reference: reproduce its modules, public +interfaces, and data models idiomatically in THIS project's language and +conventions. Do NOT copy code verbatim, and do NOT create files under the +reference path — it is not part of this project. + +### Module / symbol map (path: top-level symbols) +""" + + +def build_reference_digest( + reference_dir: str | Path, + cache_dir: str | Path | None = None, + max_chars: int = _DEFAULT_MAX_CHARS, +) -> str: + """Render a compact, read-only digest of ``reference_dir`` for porting. + + Parameters + ---------- + reference_dir: + Path to the reference implementation to analyze. Must be an existing + directory. Resolved to an absolute path (user ``~`` expanded). + cache_dir: + Where to write the symbol-graph cache. Redirected here so the reference + tree is never mutated. Defaults to a throwaway temp directory when + omitted. Its parent is created if missing. + max_chars: + Hard ceiling on the rendered symbol map (excluding the header). The map + is truncated at a line boundary with an elision note when exceeded. + + Returns + ------- + A markdown digest string. Never empty on success; raises ``ValueError`` if + the path is missing/not a directory. Extraction that yields no symbols (an + empty or unparseable tree) returns the header plus an explicit note so the + caller can still see the reference was consulted. + """ + ref = Path(reference_dir).expanduser().resolve() + if not ref.is_dir(): + raise ValueError(f"reference dir not found or not a directory: {reference_dir}") + if max_chars <= 0: + raise ValueError(f"max_chars must be > 0, got {max_chars}") + + graph = SymbolGraph(ref) + # Redirect the on-disk cache OFF the reference tree: this is the single line + # that guarantees analysis is read-only. A temp dir when no target cache is + # given; either way, never ``ref``. + if cache_dir is not None: + cache_root = Path(cache_dir) + cache_root.mkdir(parents=True, exist_ok=True) + else: + cache_root = Path(tempfile.mkdtemp(prefix="misterdev-refcache-")) + graph.cache_path = cache_root / "reference_topography_cache.json" + + graph.build() + outline = graph.project_outline() + + header = _HEADER.format(name=ref.name) + if not outline.strip(): + logger.info("Reference digest: no symbols extracted from %s", ref) + return header + "(no source symbols could be extracted from the reference)" + + if len(outline) > max_chars: + truncated = outline[:max_chars].rsplit("\n", 1)[0] + omitted = outline.count("\n") - truncated.count("\n") + outline = truncated + f"\n(... reference map truncated, ~{omitted} more lines)" + + return header + outline diff --git a/misterdev/cli.py b/misterdev/cli.py index 926ecf3..7bb82fe 100644 --- a/misterdev/cli.py +++ b/misterdev/cli.py @@ -88,6 +88,25 @@ def _print_report(project_path: str) -> None: console.print("[dim]No audit trail yet.[/]") +def _print_doctor(result: dict) -> None: + """Render the doctor checklist and return nothing (caller handles exit).""" + glyph = {"pass": "[green]✓[/]", "warn": "[yellow]⚠[/]", "fail": "[red]✗[/]"} + console.print("[bold]misterdev doctor[/] — preflight for an unattended run\n") + for c in result.get("checks", []): + console.print(f"{glyph.get(c.status, '?')} {c.name}[dim] — {c.detail}[/]") + if c.status != "pass" and c.fix: + console.print(f" [dim]fix:[/] {c.fix}") + verdict = ( + "[bold green]READY[/]" + if result.get("ready") + else "[bold red]NOT READY[/] (hard blocker present)" + ) + console.print( + f"\n{verdict} · {result.get('passed', 0)} ok · " + f"{result.get('warnings', 0)} warning(s) · {result.get('failures', 0)} failure(s)" + ) + + def main(): # Natural-language mode: when the first argument isn't a known subcommand # (and isn't a flag), treat the whole line as plain English and let @@ -100,6 +119,7 @@ def main(): "list", "status", "report", + "doctor", "run", "plan", "build", @@ -142,6 +162,16 @@ def main(): "project_path", type=str, nargs="?", default=".", help="Path to the project" ) + # 'doctor' command — preflight a project before an unattended run + doctor_parser = subparsers.add_parser( + "doctor", + help="Preflight a project for an unattended run (clean tree, models, " + "worktree prime, requirements); exits non-zero on a hard blocker", + ) + doctor_parser.add_argument( + "project_path", type=str, nargs="?", default=".", help="Path to the project" + ) + # 'run' command (legacy) run_parser = subparsers.add_parser("run", help="Run tasks for a project") run_parser.add_argument( @@ -273,6 +303,13 @@ def main(): default=None, help="Cap the number of tasks this run will plan/execute (bounds cost)", ) + build_parser.add_argument( + "--reference", + type=str, + default=None, + help="Path to a reference implementation to port from (analyzed " + "read-only; its module/symbol map guides the plan)", + ) # 'mcp' — serve misterdev as an MCP server over stdio (same as the # misterdev-mcp console script; exposed as a subcommand so runners like uvx @@ -332,6 +369,10 @@ def main(): console.print(table) elif args.command == "report": _print_report(args.project_path) + elif args.command == "doctor": + result = orchestrator.run_doctor(args.project_path) + _print_doctor(result) + sys.exit(result.get("exit_code", 1)) elif args.command == "run": if args.task: logger.info( @@ -375,7 +416,9 @@ def main(): if args.max_tasks is not None: build_args.extend(["--max-tasks", str(args.max_tasks)]) - report = orchestrator.build(args.project_path, " ".join(build_args)) + report = orchestrator.build( + args.project_path, " ".join(build_args), reference_dir=args.reference + ) console.print("\n") if orchestrator.last_build_succeeded: console.print( diff --git a/misterdev/config.py b/misterdev/config.py index 1fef654..ec9d3d8 100644 --- a/misterdev/config.py +++ b/misterdev/config.py @@ -262,15 +262,15 @@ class OrchestratorSettings: goal_check: bool = False block_on_goal_gap: bool = False goal_check_timeout: int = 60 - # Spec-as-tests (CONSERVATIVE, opt-in, currently DEFERRED): generate a failing - # test from a task's acceptance criteria before it is implemented. Off by - # default. The generation primitive lives in core/spec_tests.py and is tested, - # but it is NOT wired into the execute loop yet: writing a failing test inside - # the wave loop would flip the integration-gate baseline red and silently - # disable that gate, which is not control-flow-neutral. When set true today it - # only logs that the feature is staged-but-not-wired (see the seam in - # core/spec_tests.py); it never alters the build loop. - spec_as_tests: bool = False + # Spec-as-tests (reproduction-first / TDD): before a task is implemented, + # generate a failing test from its acceptance criteria so "done" means "this + # test now passes", not a self-report. DEFAULT-ON. Wired in the executor + # (_maybe_generate_spec_test / _run_spec_test): the test is written under + # .orchestrator/spec_tests/ — OUTSIDE the project suite — so it can never flip + # the integration-gate baseline red, and it is run scoped to that one file + # after the task's own gates pass. ADVISORY unless spec_as_tests_block is set. + # Generation is best-effort and timeout-bounded. + spec_as_tests: bool = True # When spec_as_tests is on, a per-task generated spec test is run (scoped, # from .orchestrator/spec_tests/) after the task's gates pass. ADVISORY by # default: a still-failing spec test is logged/recorded but does not fail the @@ -298,12 +298,78 @@ class OrchestratorSettings: # working tree), or "worktree" (always isolate each parallel task). parallel_mode: str = "auto" # Run independent tasks within a wave concurrently in `run --tasks` (they are - # worktree-isolated on a git repo; `max_workers`/`parallel_mode` apply). Gates - # resolve deps natively in a worktree (pnpm/npm populate node_modules from their - # store on first run — verified; do NOT symlink node_modules, it breaks pnpm). + # worktree-isolated on a git repo; `max_workers`/`parallel_mode` apply). # Off by default because concurrent gates contend for CPU/the store; enable per # project after a live check, and keep `max_workers` modest on a laptop. run_parallel: bool = False + # Prime a fresh worktree's dependencies ONCE at setup (serially, before the + # parallel gate phase). A new git worktree has no node_modules, so an un-primed + # gate pays a full install (~15-20s) INSIDE its own timeout while N worktrees + # hammer the package store at once — which intermittently times out and fails + # correct code. Priming moves that install out of the gate and off the hot path. + # None auto-detects the command from the lockfile; "" disables; or set an + # explicit command. Do NOT symlink node_modules to skip this — it breaks pnpm. + worktree_setup_command: Optional[str] = None + worktree_setup_timeout: int = 600 + # Prime a worktree by CLONING the base checkout's node_modules (copy-on-write: + # APFS clonefile / Linux hardlinks) instead of reinstalling — near-instant vs a + # full ``pnpm install``. On by default; automatically falls back to the setup + # command on a non-CoW filesystem (e.g. HFS+) or if the clone fails the P3 + # sanity probe, so it is always safe. Set false to force a plain install. + worktree_clone_deps: bool = True + # Fast sanity probe run right after priming to confirm the worktree's + # toolchain actually resolves — a broken/partial install (a killed download, + # a locked store) otherwise masquerades as a code failure in the gate. None + # auto-detects (e.g. ``npx tsc --version`` for a node/pnpm project); "" disables; + # or set an explicit command. Cheap and bounded by ``worktree_setup_timeout``. + worktree_healthcheck_command: Optional[str] = None + # Before dispatching a ready task, skip it (mark completed) when it is already + # satisfied — its content hash is unchanged since a recorded completion + # (compute_task_hash + the committed ledger). Avoids spawning a worktree and + # paying a full prime/install just to re-recognize done work. On by default; + # set false to force every ready task to re-run regardless of the ledger. No + # gates are run on the base branch for this decision (that would serialize and + # contend); it rests purely on the content hash and the ledger. + skip_satisfied_tasks: bool = True + # Split a parallel wave into conflict-free sub-waves: tasks that DECLARE a + # shared file (env.ts, a route registry, a schema) run in different sub-waves + # (serially) so their worktree merges can't race or clobber; disjoint tasks + # stay parallel. On by default. Set false to run every ready task in one wave. + serialize_conflicting_tasks: bool = True + # Escalation ladder for a task that keeps failing (real code failures only — + # infra faults self-heal and never advance it). After N non-infra failures the + # executor climbs: widen context -> stronger model -> request decomposition. + # Bounded and config-driven; thresholds are cumulative code-failure counts. + escalation_enabled: bool = True + escalation_widen_after: int = 1 + # full_rewrite rung: after this many code failures, stop patching and rewrite + # the whole target region from scratch — a structurally different attempt. + escalation_rewrite_after: int = 2 + # model/decompose coincide at 3 by default so a 3-attempt task climbs + # normal -> widen -> full_rewrite and decomposes at exhaustion; raise + # decompose_after to make the stronger_model rung reachable. + escalation_model_after: int = 3 + escalation_decompose_after: int = 3 + # The stronger model id to route generation to at the model rung (None -> keep + # the routed/default model, i.e. that rung only widens context). + escalation_model: Optional[str] = None + # After each successful worktree merge into the base branch, run the merged + # task's owning-target gate (typecheck/test) on the base checkout. If it fails + # with a real (non-infra) error the merge broke the base, so roll it back + # (reset the merge commit) and treat the task as unfinished — the base branch + # is never left red at wave end. On by default; a transient/infra failure is + # NOT rolled back (it is an environment fault, not a code break). + post_merge_healthcheck: bool = True + # Adapt concurrency/timeouts to ENVIRONMENT faults between waves: when a wave's + # infra-failure count exceeds ``adaptive_infra_threshold``, the NEXT wave halves + # effective max_workers (floor 1) and multiplies gate/setup timeouts by + # ``adaptive_timeout_factor`` (bounded by ``adaptive_max_timeout_factor``); + # clean waves recover gradually toward the configured values. On by default — + # it only engages under sustained contention and is otherwise a no-op. + adaptive_concurrency: bool = True + adaptive_infra_threshold: int = 1 + adaptive_timeout_factor: float = 2.0 + adaptive_max_timeout_factor: float = 4.0 auto_detect_dependencies: bool = False # Optional MCP (Model Context Protocol) tool awareness: when on, the tools # discovered from the servers in the top-level ``mcp.servers`` list are diff --git a/misterdev/core/context/topography/graph.py b/misterdev/core/context/topography/graph.py index 2e102b3..7a5b4fe 100644 --- a/misterdev/core/context/topography/graph.py +++ b/misterdev/core/context/topography/graph.py @@ -2,7 +2,7 @@ import os from pathlib import Path -from typing import Dict, List, Optional, Set, Any +from typing import Any, Dict, List, Optional, Set, Tuple from misterdev.utils.file_utils import read_file, is_golden_path @@ -665,12 +665,27 @@ def _resolve_references(self): symbol.outgoing_calls.add(other_key) self.symbols[other_key].incoming_calls.add(key) + def _file_index(self) -> Dict[str, List[Tuple[str, SymbolNode]]]: + """Per-file ``[(key, node)]`` index, memoized, sorted by start line. + + The file-scoped queries below were each a full O(all-symbols) scan; on a + large repo that is O(errors x symbols) / O(files x symbols) across a run. + The index is rebuilt only when ``self.symbols`` is replaced or resized + (``build()`` swaps the whole dict), keyed on ``(id, len)``.""" + key = (id(self.symbols), len(self.symbols)) + if getattr(self, "_file_index_key", None) != key: + idx: Dict[str, List[Tuple[str, SymbolNode]]] = {} + for skey, s in self.symbols.items(): + idx.setdefault(s.file_path, []).append((skey, s)) + for entries in idx.values(): + entries.sort(key=lambda kv: kv[1].start_line) + self._file_index_cache = idx + self._file_index_key = key + return self._file_index_cache + def file_symbols(self, file_path: str) -> List[SymbolNode]: """Symbols defined in one file, ordered by start line.""" - return sorted( - (s for s in self.symbols.values() if s.file_path == file_path), - key=lambda s: s.start_line, - ) + return [s for _key, s in self._file_index().get(file_path, [])] def _match_files(self, file_path: str) -> Set[str]: """Graph file paths an error path refers to: an exact match when present, @@ -678,13 +693,11 @@ def _match_files(self, file_path: str) -> Set[str]: cwd (``src/app.ts``) still resolves against the root-relative key (``frontend/src/app.ts``). An ambiguous suffix yields no match rather than a wrong one.""" - exact = {s.file_path for s in self.symbols.values() if s.file_path == file_path} - if exact: - return exact + idx = self._file_index() + if file_path in idx: + return {file_path} suffix = "/" + file_path - matches = { - s.file_path for s in self.symbols.values() if s.file_path.endswith(suffix) - } + matches = {f for f in idx if f.endswith(suffix)} return matches if len(matches) == 1 else set() def symbol_at_line(self, file_path: str, line: int) -> Optional[str]: @@ -699,16 +712,16 @@ def symbol_at_line(self, file_path: str, line: int) -> Optional[str]: if not files: return None row = line - 1 + idx = self._file_index() best_key: Optional[str] = None best_span: Optional[int] = None - for key, sym in self.symbols.items(): - if sym.file_path not in files: - continue - if sym.start_line <= row <= sym.end_line: - span = sym.end_line - sym.start_line - if best_span is None or span < best_span: - best_span = span - best_key = key + for f in files: + for skey, sym in idx.get(f, []): + if sym.start_line <= row <= sym.end_line: + span = sym.end_line - sym.start_line + if best_span is None or span < best_span: + best_span = span + best_key = skey return best_key def callers_of(self, key: str) -> List[str]: diff --git a/misterdev/core/economics/context_budget.py b/misterdev/core/economics/context_budget.py index 3331356..853a9c2 100644 --- a/misterdev/core/economics/context_budget.py +++ b/misterdev/core/economics/context_budget.py @@ -76,11 +76,22 @@ def __init__(self, max_tokens: int = 100000, reserved_tokens: int = 8000): self.available = max_tokens - reserved_tokens self._sections: Dict[str, _Section] = {} - def set(self, name: str, content: str, priority: int = 2, min_lines: int = 10): + def set( + self, + name: str, + content: str, + priority: int = 2, + min_lines: int = 10, + truncatable: bool = True, + ): """Register a context section. priority: 1 = essential (truncate last), 2 = important, 3 = nice-to-have (truncate first) min_lines: minimum lines to keep even under pressure + truncatable: when False the section is kept VERBATIM even under budget + pressure (never trimmed, no elision marker). Use for the edit region — + the exact lines a SEARCH/REPLACE edit must match — where a dropped tail + makes the model edit blind against spans it can no longer see. """ self._sections[name] = _Section( name=name, @@ -88,6 +99,7 @@ def set(self, name: str, content: str, priority: int = 2, min_lines: int = 10): priority=priority, min_lines=min_lines, tokens=estimate_tokens(content), + truncatable=truncatable, ) def allocate(self) -> Dict[str, str]: @@ -116,6 +128,14 @@ def allocate(self) -> Dict[str, str]: result[section.name] = section.content continue + # The edit region (and anything marked truncatable=False) is kept + # verbatim even under pressure: a dropped tail makes the model edit + # blind against spans it can no longer see. Truncatable sections then + # absorb the overflow instead. + if not section.truncatable: + result[section.name] = section.content + continue + # How much can we trim from this section? lines = section.content.splitlines() if len(lines) <= section.min_lines: @@ -148,13 +168,20 @@ def summary(self) -> str: class _Section: def __init__( - self, name: str, content: str, priority: int, min_lines: int, tokens: int + self, + name: str, + content: str, + priority: int, + min_lines: int, + tokens: int, + truncatable: bool = True, ): self.name = name self.content = content self.priority = priority self.min_lines = min_lines self.tokens = tokens + self.truncatable = truncatable def _truncate_to_tokens(lines: list, target_tokens: int, name: str) -> str: diff --git a/misterdev/core/economics/model_selector.py b/misterdev/core/economics/model_selector.py index 301bd67..fc2dd57 100644 --- a/misterdev/core/economics/model_selector.py +++ b/misterdev/core/economics/model_selector.py @@ -217,9 +217,32 @@ def select( # final=True lifts the latency guard here (a slow-but-capable last resort # beats giving up), but incompetent models are still excluded. if attempt >= max_attempts - 1: - return self._pick( + pick = self._pick( self._tier_models(rungs[last]), category, complexity, True, final=True ) + if pick is not None: + return pick + # The strongest tier is empty for this cell (every model in it is proven + # incompetent). Do NOT silently fall back to the client default on the + # last resort — widen to the best usable model across ALL tiers and warn. + wider = [m for r in rungs for m in self._tier_models(r)] + pick = self._pick(wider, category, complexity, True, final=True) + if pick is not None: + logger.warning( + "Model selector: strongest tier has no usable model for (%s, %s) " + "on the final attempt; using %s from a wider set.", + category, + complexity, + pick, + ) + else: + logger.warning( + "Model selector: every configured model is excluded for (%s, %s) " + "on the final attempt; falling back to the client default.", + category, + complexity, + ) + return pick if attempt == 0 and not self._explore_on_first(category, complexity): # Conservative first attempt: cheapest rung with a proven model, else @@ -275,13 +298,3 @@ def _announce_graduation(self, category: str, complexity: str, model: str) -> No f"[auto] {category}/{complexity} now using proven cheap model " f"{model!r} on first attempt" ) - - def is_ready(self, category: str, complexity: str) -> bool: - """Whether this cell would use a proven cheap model on a first attempt.""" - if not self.enabled: - return False - return any( - self._proven(m, category, complexity) - for tier in self._rungs()[:-1] - for m in self._tier_models(tier) - ) diff --git a/misterdev/core/evolution/scheduled.py b/misterdev/core/evolution/scheduled.py new file mode 100644 index 0000000..fce609e --- /dev/null +++ b/misterdev/core/evolution/scheduled.py @@ -0,0 +1,63 @@ +"""Schedulable entrypoint for one live evolution pass, gated behind the benchmark. + +`run_evolution(..., live=True)` already promotes a self-edit ONLY when it beats the +champion on the benchmark. What was missing is a component that decides WHEN to run +it: a callable an external scheduler (cron/CI/`/loop`) can invoke, guarded by an +exclusive lock so overlapping scheduled runs cannot corrupt the shared archive or +worktree. A busy lock is a clean no-op. +""" + +import os +from pathlib import Path +from typing import Callable, Optional + +from misterdev.core.evolution.driver import run_evolution +from misterdev.logging_setup import setup_logger + +logger = setup_logger(__name__) + + +def run_scheduled_evolution( + project, + benchmark_dir: str, + workdir: str, + *, + gate_commands: dict, + lock_path: Optional[str] = None, + _run: Callable = run_evolution, + **kwargs, +): + """Run ONE live, benchmark-gated evolution pass under an exclusive lock. + + Returns the :class:`EvolutionResult`, or ``None`` when another scheduled pass + already holds the lock (overlap is skipped, never queued). ``_run`` is the + evolution entrypoint, injectable for tests. Always ``live=True`` — a scheduled + pass only ever applies a change that passes the benchmark gate. + """ + lock = Path( + lock_path + or (Path(project.path) / ".orchestrator" / "evolution" / "scheduled.lock") + ) + lock.parent.mkdir(parents=True, exist_ok=True) + try: + fd = os.open(str(lock), os.O_CREAT | os.O_EXCL | os.O_WRONLY) + except FileExistsError: + logger.warning( + "Scheduled evolution: another pass holds %s; skipping this trigger.", lock + ) + return None + try: + return _run( + project, + benchmark_dir, + workdir, + gate_commands=gate_commands, + live=True, + **kwargs, + ) + finally: + os.close(fd) + try: + lock.unlink() + except OSError: + pass diff --git a/misterdev/core/execution/adaptive.py b/misterdev/core/execution/adaptive.py new file mode 100644 index 0000000..3a19ff2 --- /dev/null +++ b/misterdev/core/execution/adaptive.py @@ -0,0 +1,68 @@ +"""Adaptive concurrency/timeout backoff on infrastructure faults. + +When a wave shows repeated ENVIRONMENT faults (see ``infra.py``) — gate timeouts, +locked package stores, OOM — the machine is contended, not the code broken. +Thrashing on at full concurrency makes it worse: every parallel gate competes for +the same CPU/store and times out. The right response is to back off — fewer +concurrent workers, longer timeouts — for the NEXT wave, then recover gradually +once waves come back clean. + +This module holds ONLY the pure decision: given the just-finished wave's infra +count and the current (already-adapted) settings, what should the next wave use? +Keeping it side-effect-free makes the backoff/recovery policy directly unit +testable; the orchestrator owns measuring the count and applying the result. +""" + +from dataclasses import dataclass + + +@dataclass(frozen=True) +class WaveTuning: + """The two knobs the orchestrator adapts between waves. + + ``max_workers`` is the concurrency cap; ``timeout_factor`` multiplies the + configured gate/setup timeouts (1.0 == the configured values). + """ + + max_workers: int + timeout_factor: float + + +def next_wave_tuning( + infra_count: int, + current: WaveTuning, + *, + base_workers: int, + threshold: int = 1, + timeout_factor: float = 2.0, + max_timeout_factor: float = 4.0, + min_workers: int = 1, +) -> WaveTuning: + """Next wave's tuning from the last wave's infra count and current settings. + + - Infra count ABOVE ``threshold``: back off — halve workers (floor + ``min_workers``) and grow the timeout factor by ``timeout_factor`` (ceiling + ``max_timeout_factor``). + - A CLEAN wave (count 0): recover gradually toward the configured values — + double workers (ceiling ``base_workers``) and shrink the timeout factor by + ``timeout_factor`` (floor 1.0). + - Nonzero but AT/BELOW ``threshold``: hold steady, so a single self-healed + hiccup neither panics nor prematurely recovers. + + Pure and idempotent at the bounds: at full concurrency a clean wave stays at + full concurrency; at the floor a bad wave stays at the floor. + """ + base_workers = max(1, int(base_workers)) + floor = max(1, int(min_workers)) + ceil = max(1.0, float(max_timeout_factor)) + step = max(1.0, float(timeout_factor)) + cur_workers = max(floor, int(current.max_workers)) + cur_factor = min(ceil, max(1.0, float(current.timeout_factor))) + + if infra_count > threshold: + return WaveTuning(max(floor, cur_workers // 2), min(ceil, cur_factor * step)) + if infra_count == 0: + return WaveTuning( + min(base_workers, cur_workers * 2), max(1.0, cur_factor / step) + ) + return WaveTuning(cur_workers, cur_factor) diff --git a/misterdev/core/execution/blocker.py b/misterdev/core/execution/blocker.py index 80be8bd..5d3c2de 100644 --- a/misterdev/core/execution/blocker.py +++ b/misterdev/core/execution/blocker.py @@ -33,19 +33,23 @@ "a Cloudflare account_id is required", ), # Credential-ish terms (env-var style ``_TOKEN`` / ``_KEY`` included) paired - # with a "needed" word — but NOT bare "token", so a parser's "unexpected - # token" / "invalid token" is not misread as a credential block. + # with an ABSENCE word only — never "invalid"/"expired", which are validation + # RESULTS a normal test emits ("expected api key to be valid, got invalid" is a + # feature test, not a missing-credential block; a genuinely bad credential + # surfaces as 401/unauthorized below). This keeps a task that IMPLEMENTS + # api-key issuance from being misread as needing an external key. ( re.compile( r"(?:api[_ ]?key|api[_ ]?token|access[_ ]?token|auth[_ ]?token|" r"bearer\s+token|_token\b|_key\b|secret|credentials?)" - r".{0,30}(?:required|missing|not set|unset|undefined|invalid|empty|expired)|" - r"(?:required|missing|not set|unset|undefined|invalid|empty|expired)" - r".{0,30}(?:api[_ ]?key|api[_ ]?token|access[_ ]?token|auth[_ ]?token|" - r"_token\b|_key\b|secret|credentials?)", + r".{0,30}(?:is required|are required|required\b|missing(?!\s+from)|" + r"not set|not configured|not provided|unset|undefined)|" + r"(?:required|missing|no|unset|undefined|set the|provide (?:a|the|your))" + r"\s.{0,20}(?:api[_ ]?key|api[_ ]?token|access[_ ]?token|auth[_ ]?token|" + r"_token\b|_key\b|credentials?)", re.I, ), - "a required API key / token / secret is missing or invalid", + "a required API key / token / secret is missing", ), ( re.compile( @@ -88,11 +92,36 @@ ) +# A bare 401/403 emitted INSIDE a test run is NOT a missing-credential block: it is +# either a test asserting on a 401/403 response, or local harness noise (a +# workers/miniflare pool probe). Parking the task there freezes a whole subtree on +# a "needs your credential" that the code should just fix (observed: a countless +# server keystone gated by @cloudflare/vitest-pool-workers). So the two BROAD +# HTTP-status auth rules are suppressed under test context; the SPECIFIC rules +# (an explicit `wrangler login`, a named missing key/token) still fire — those +# name a real external resource, not an HTTP status a test can legitimately emit. +_TEST_CONTEXT = re.compile( + r"\bvitest\b|\bjest\b|\bminiflare\b|vitest-pool-workers|" + r"\bexpect\(|\.toBe\b|\.toEqual\b|\bdescribe\(|\bit\(|" + r"\bTest Files\b|\.test\.|\.spec\.", + re.I, +) +_TEST_SUPPRESSIBLE = frozenset( + { + "authentication failed (401 / invalid credentials)", + "access forbidden (403) — the account lacks permission", + } +) + + def blocked_reason(output: str) -> Optional[str]: """A short human reason when ``output`` shows an external-resource block, else None. None means "treat as an ordinary (retryable / real) failure".""" text = output or "" + in_test = bool(_TEST_CONTEXT.search(text)) for pattern, reason in _SIGNALS: if pattern.search(text): + if in_test and reason in _TEST_SUPPRESSIBLE: + continue # a 401/403 inside a test run is code to fix, not a block return reason return None diff --git a/misterdev/core/execution/compile_view.py b/misterdev/core/execution/compile_view.py index 3a3cbaf..0c6d80f 100644 --- a/misterdev/core/execution/compile_view.py +++ b/misterdev/core/execution/compile_view.py @@ -17,7 +17,7 @@ import re from dataclasses import dataclass -from typing import List, Optional +from typing import Callable, Dict, List, Optional @dataclass @@ -30,6 +30,42 @@ class CompileError: detail: Optional[str] = None # e.g. "expected `u32`, found `&str`" +@dataclass(frozen=True) +class CompilerAdapter: + """A per-language compiler-diagnostic adapter. + + ``language`` is the canonical lowercase language key ("rust", "typescript"). + ``name`` is the tool the parser targets ("rustc", "tsc"). ``parse`` turns raw + compiler output into ``CompileError`` records (empty when it recognizes + nothing). ``detect`` reports whether this adapter's compiler produced the + output, so a caller with no language hint can route by content. New languages + register an adapter instead of extending an if/else on language name. + """ + + language: str + name: str + parse: Callable[[str], List[CompileError]] + detect: Callable[[str], bool] + + +_REGISTRY: Dict[str, CompilerAdapter] = {} + + +def register_adapter(adapter: CompilerAdapter) -> None: + """Register (or replace) the adapter for ``adapter.language``.""" + _REGISTRY[adapter.language] = adapter + + +def get_adapter(language: Optional[str]) -> Optional[CompilerAdapter]: + """The adapter for a language key, or None when none is registered.""" + return _REGISTRY.get((language or "").lower()) + + +def registered_languages() -> List[str]: + """The languages that currently have a registered adapter.""" + return sorted(_REGISTRY) + + # --- rustc ------------------------------------------------------------------ # `error[E0308]: mismatched types` / `error: cannot find value ...` (no code). @@ -70,31 +106,155 @@ def _parse_rustc(output: str) -> List[CompileError]: return out -_COMPILERS = {"rustc": _parse_rustc} +def _detect_rustc(output: str) -> bool: + return bool(re.search(r"^error(?:\[E\d+\])?: ", output, re.M)) and "-->" in output + + +# --- tsc (TypeScript) ------------------------------------------------------- + +# Classic tsc: `src/app.ts(12,5): error TS2322: Type 'string' is not ...` +_TSC_CLASSIC = re.compile( + r"^(?P.+?)\((?P\d+),(?P\d+)\):\s*" + r"error\s+(?PTS\d+):\s*(?P.+)$" +) +# Pretty tsc (`--pretty`): `src/app.ts:12:5 - error TS2322: Type 'string' ...` +_TSC_PRETTY = re.compile( + r"^(?P\S+):(?P\d+):(?P\d+)\s*-\s*" + r"error\s+(?PTS\d+):\s*(?P.+)$" +) + + +def _parse_tsc(output: str) -> List[CompileError]: + out: List[CompileError] = [] + for line in output.splitlines(): + m = _TSC_CLASSIC.match(line) or _TSC_PRETTY.match(line) + if not m: + continue + out.append( + CompileError( + message=m.group("message").strip(), + code=m.group("code"), + location=f"{m.group('file').strip()}:{m.group('line')}:{m.group('col')}", + ) + ) + return out + + +def _detect_tsc(output: str) -> bool: + return bool(re.search(r"error\s+TS\d+:", output)) + + +# --- go (go build / go vet) ------------------------------------------------- + +# `./main.go:10:6: undefined: helper` — the column is optional. +_GO_ERROR = re.compile( + r"^(?P\S+\.go):(?P\d+):(?:(?P\d+):)?\s+(?P.+)$" +) + + +def _parse_go(output: str) -> List[CompileError]: + out: List[CompileError] = [] + for line in output.splitlines(): + m = _GO_ERROR.match(line) + if not m: + continue + loc = f"{m.group('file')}:{m.group('line')}" + if m.group("col"): + loc += f":{m.group('col')}" + out.append(CompileError(message=m.group("message").strip(), location=loc)) + return out + + +def _detect_go(output: str) -> bool: + return bool(re.search(r"^\S+\.go:\d+:", output, re.M)) + + +# --- swiftc ----------------------------------------------------------------- + +# `/src/App.swift:12:15: error: cannot find 'foo' in scope` +_SWIFT_ERROR = re.compile( + r"^(?P.+?\.swift):(?P\d+):(?P\d+):\s*error:\s*(?P.+)$" +) + + +def _parse_swift(output: str) -> List[CompileError]: + out: List[CompileError] = [] + for line in output.splitlines(): + m = _SWIFT_ERROR.match(line) + if not m: + continue + out.append( + CompileError( + message=m.group("message").strip(), + location=f"{m.group('file')}:{m.group('line')}:{m.group('col')}", + ) + ) + return out + + +def _detect_swift(output: str) -> bool: + return bool(re.search(r"\.swift:\d+:\d+:\s*error:", output)) + + +# --- csc / dotnet (MSBuild) ------------------------------------------------- + +# `Program.cs(12,20): error CS0103: ` with an optional trailing +# ` [project.csproj]` MSBuild tag, which is stripped from the message. +_CSC_ERROR = re.compile( + r"^(?P.+?)\((?P\d+),(?P\d+)\):\s*error\s+(?PCS\d+):\s*" + r"(?P.+?)(?:\s*\[[^\]]+\])?$" +) -def _detect_compiler(output: str) -> Optional[str]: - if re.search(r"^error(?:\[E\d+\])?: ", output, re.M) and "-->" in output: - return "rustc" - return None +def _parse_csharp(output: str) -> List[CompileError]: + out: List[CompileError] = [] + for line in output.splitlines(): + m = _CSC_ERROR.match(line) + if not m: + continue + out.append( + CompileError( + message=m.group("message").strip(), + code=m.group("code"), + location=f"{m.group('file')}:{m.group('line')}:{m.group('col')}", + ) + ) + return out + + +def _detect_csharp(output: str) -> bool: + return bool(re.search(r"error\s+CS\d+:", output)) + + +register_adapter(CompilerAdapter("rust", "rustc", _parse_rustc, _detect_rustc)) +register_adapter(CompilerAdapter("typescript", "tsc", _parse_tsc, _detect_tsc)) +register_adapter(CompilerAdapter("go", "go build", _parse_go, _detect_go)) +register_adapter(CompilerAdapter("swift", "swiftc", _parse_swift, _detect_swift)) +register_adapter(CompilerAdapter("csharp", "csc", _parse_csharp, _detect_csharp)) def extract_compile_errors( output: str, language: Optional[str] = None ) -> List[CompileError]: - """Parse compiler output into diagnostic records. Empty on unrecognized output.""" + """Parse compiler output into diagnostic records. Empty on unrecognized output. + + Routes by the explicit ``language`` when its adapter recognizes the output; + otherwise falls back to content detection across the registry, so a caller + with the wrong or no language hint still gets the right diagnostics. + """ if not output: return [] - compiler = {"rust": "rustc"}.get((language or "").lower()) - if ( - compiler is None - or compiler not in _COMPILERS - or not _COMPILERS[compiler](output) - ): - compiler = _detect_compiler(output) - if compiler is None: - return [] - return _COMPILERS[compiler](output) + adapter = get_adapter(language) + if adapter is not None: + errors = adapter.parse(output) + if errors: + return errors + for candidate in _REGISTRY.values(): + if candidate.detect(output): + errors = candidate.parse(output) + if errors: + return errors + return [] def render_compile_view(errors: List[CompileError], max_errors: int = 5) -> str: diff --git a/misterdev/core/execution/deferral.py b/misterdev/core/execution/deferral.py index a888b4a..5031fbe 100644 --- a/misterdev/core/execution/deferral.py +++ b/misterdev/core/execution/deferral.py @@ -54,7 +54,7 @@ def load_answers(self) -> Dict[str, str]: current: str = "" for line in lines: if line.startswith("## "): - current = line[3:].split("—", 1)[0].split(" - ", 1)[0].strip() + current = line[3:].split("—", 1)[0].strip() continue low = line.strip() marker = "- answer:" diff --git a/misterdev/core/execution/dep_clone.py b/misterdev/core/execution/dep_clone.py new file mode 100644 index 0000000..a744b6a --- /dev/null +++ b/misterdev/core/execution/dep_clone.py @@ -0,0 +1,164 @@ +"""Prime a git worktree's dependencies by CLONING the base checkout, not reinstalling. + +Priming a fresh worktree with a full ``pnpm install`` costs ~15-20s per worktree +per wave. The base checkout already holds a fully resolved ``node_modules``; on a +copy-on-write filesystem that tree can be duplicated into the worktree near- +instantly (APFS ``clonefile(2)`` clones a whole directory hierarchy in ONE +syscall — measured at sub-millisecond — sharing storage until a file is written). + +Why the symlinks stay correct: a git worktree is a checkout of the SAME repo at +the SAME paths, and pnpm links workspace packages with RELATIVE symlinks +(``apps/server/node_modules/@scope/pkg -> ../../../../packages/pkg``). Cloned +verbatim, that relative link resolves against the WORKTREE, so it points at the +worktree's own package — not the base — with no reinstall and no cross-tree bleed. + +Everything here is fail-safe: ``clone_supported`` returns False on any non-CoW +filesystem (e.g. HFS+, where ``clonefile`` is ENOTSUP), and ``clone_dependencies`` +returns False on the first clone that fails. The caller then runs the normal +install, and the P3 sanity probe still gates whatever ends up in the worktree — +so a wrong guess costs an install, never a broken build. +""" + +import ctypes +import ctypes.util +import os +import platform +import shutil +import subprocess +from pathlib import Path +from typing import List, Tuple + +from misterdev.logging_setup import setup_logger + +logger = setup_logger(__name__) + +# node_modules lives at the repo root and under each workspace package; scanning +# deeper than this (or into these dirs) only finds nested deps that a parent clone +# already carries. +_MAX_DEPTH = 6 +_SKIP = {".git", ".orchestrator", ".hg", ".svn", "node_modules"} + + +def _macos_clonefile(src: str, dst: str) -> None: + """Raw ``clonefile(2)``: recursive CoW clone of ``src`` to ``dst`` (which must + not exist). Raises OSError on failure — notably ENOTSUP on a non-APFS volume, + which the caller treats as "cloning unavailable, install instead".""" + libc = ctypes.CDLL(ctypes.util.find_library("c"), use_errno=True) + libc.clonefile.argtypes = [ctypes.c_char_p, ctypes.c_char_p, ctypes.c_uint32] + if libc.clonefile(os.fsencode(src), os.fsencode(dst), 0) != 0: + errno = ctypes.get_errno() + raise OSError(errno, os.strerror(errno), dst) + + +def _clone_tree(src: Path, dst: Path) -> None: + """Clone one directory tree ``src`` -> ``dst`` (``dst`` must not exist). + + macOS uses ``clonefile`` (recursive, one syscall). Linux uses ``cp -al`` + (hardlinked archive copy, same-filesystem). Raises OSError on failure. + """ + system = platform.system() + if system == "Darwin": + _macos_clonefile(str(src), str(dst)) + elif system == "Linux": + r = subprocess.run( + ["cp", "-al", "--", str(src), str(dst)], + capture_output=True, + text=True, + ) + if r.returncode != 0: + raise OSError(r.returncode, (r.stderr or "cp -al failed").strip()) + else: + raise OSError(f"dependency clone unsupported on {system}") + + +def clone_supported(base) -> bool: + """Whether ``base``'s filesystem supports a fast CoW/hardlink clone. + + Probes with a single tiny file — a raw ``clonefile`` on macOS (ENOTSUP on + HFS+), a hardlink on Linux — so a non-CoW volume is detected up front and the + caller never pays a slow full-copy masquerading as a clone. Never raises. + """ + system = platform.system() + if system not in ("Darwin", "Linux"): + return False + probe_dir = Path(base) / ".orchestrator" + src = probe_dir / ".cloneprobe_src" + dst = probe_dir / ".cloneprobe_dst" + try: + probe_dir.mkdir(parents=True, exist_ok=True) + src.write_bytes(b"x") + if dst.exists() or dst.is_symlink(): + dst.unlink() + if system == "Darwin": + _macos_clonefile(str(src), str(dst)) + else: + os.link(src, dst) # same-fs hardlink; EXDEV/ENOTSUP -> unsupported + return dst.exists() + except OSError: + return False + finally: + for p in (src, dst): + try: + p.unlink() + except OSError: + pass + + +def find_node_modules_dirs(base, max_depth: int = _MAX_DEPTH) -> List[Path]: + """Relative paths of every ``node_modules`` dir to clone: the repo root's and + each workspace package's. Prunes descent into ``node_modules`` (its nested deps + ride along with the parent clone) and VCS/state dirs. Returns paths relative to + ``base``, so they can be recreated under the worktree.""" + base = Path(base) + found: List[Path] = [] + + def walk(d: Path, depth: int) -> None: + if depth > max_depth: + return + try: + entries = list(os.scandir(d)) + except OSError: + return + for e in entries: + if e.is_symlink() or not e.is_dir(): + continue + if e.name == "node_modules": + found.append(Path(e.path).relative_to(base)) + continue # nested deps come with the parent clone + if e.name in _SKIP: + continue + walk(Path(e.path), depth + 1) + + walk(base, 0) + return found + + +def clone_dependencies(base, worktree) -> Tuple[bool, List[str]]: + """Clone every base ``node_modules`` dir into ``worktree`` at the same path. + + Returns ``(ok, cloned_rel_paths)``. ``ok`` is False (with whatever was cloned + so far) when there is nothing to clone or ANY clone fails — the caller then + falls back to a normal install. A fresh worktree has no ``node_modules`` (it is + gitignored), but any stale destination is removed first so the clone is clean. + """ + base, worktree = Path(base), Path(worktree) + rels = find_node_modules_dirs(base) + if not rels: + return False, [] + cloned: List[str] = [] + for rel in rels: + src, dst = base / rel, worktree / rel + if not src.is_dir(): + continue + try: + dst.parent.mkdir(parents=True, exist_ok=True) + if dst.exists() or dst.is_symlink(): + shutil.rmtree(dst, ignore_errors=True) + _clone_tree(src, dst) + except OSError as e: + logger.info( + f"Dependency clone failed at {rel} ({e}); falling back to install." + ) + return False, cloned + cloned.append(str(rel)) + return bool(cloned), cloned diff --git a/misterdev/core/execution/doctor.py b/misterdev/core/execution/doctor.py new file mode 100644 index 0000000..e3bd1ba --- /dev/null +++ b/misterdev/core/execution/doctor.py @@ -0,0 +1,172 @@ +"""Preflight checks for an unattended run — the ``misterdev doctor`` command. + +An autonomous run spends real budget and edits real code, so it is worth +confirming the project is actually ready BEFORE it starts: the tree is clean and +on the base branch, no leftover ``task/*`` branches or dangling worktrees, the +configured models resolve, a throwaway worktree primes and its toolchain +resolves, and any REQUIREMENTS.md inputs are answered. + +Each check yields a ``Check(name, status, detail, fix)`` where status is +``pass`` / ``warn`` / ``fail``. Only a HARD blocker (something an unattended run +cannot recover from — a dirty tree, a stranded branch, an unroutable model) is a +``fail``; degradations that the run self-heals or falls back on are ``warn``. +``aggregate`` turns the checks into counts and the process exit code (non-zero +iff any hard failure). The check builders take already-gathered inputs, so the +routing and the aggregation are pure and directly unit-testable; the orchestrator +gathers the inputs and the CLI renders the checklist. +""" + +from dataclasses import dataclass +from typing import Dict, List, Optional + +PASS = "pass" +WARN = "warn" +FAIL = "fail" + + +@dataclass(frozen=True) +class Check: + name: str + status: str + detail: str = "" + fix: str = "" + + +def aggregate(checks: List[Check]) -> Dict: + """Counts + process exit code for a checklist. Exit is non-zero iff any check + is a hard ``fail`` — warnings never block an unattended run, they inform it.""" + passed = sum(1 for c in checks if c.status == PASS) + warnings = sum(1 for c in checks if c.status == WARN) + failures = sum(1 for c in checks if c.status == FAIL) + return { + "passed": passed, + "warnings": warnings, + "failures": failures, + "ready": failures == 0, + "exit_code": 1 if failures else 0, + } + + +def check_git_repo(is_git: bool) -> Check: + if is_git: + return Check("git repository", PASS, "project is a git repo") + return Check( + "git repository", + WARN, + "not a git repo — parallel worktree isolation is unavailable", + "run `git init` (and commit a baseline) for isolated parallel execution", + ) + + +def check_clean_tree(dirty: str) -> Check: + if not dirty: + return Check("clean working tree", PASS, "no uncommitted changes") + return Check( + "clean working tree", + FAIL, + f"uncommitted changes present: {dirty[:120]}", + "commit or stash your changes (a run carries/reverts branch-per-task work)", + ) + + +def check_on_base_branch(current: Optional[str], base: str) -> Check: + if not current or current in ("HEAD", "(detached)"): + return Check( + "on base branch", + FAIL, + "HEAD is detached", + f"`git checkout {base}` so tasks branch from and merge into the base", + ) + if current.startswith("task/") or current.startswith("doctor/"): + return Check( + "on base branch", + FAIL, + f"HEAD is on a leftover run branch '{current}'", + f"`git checkout {base}` and delete the stranded branch", + ) + if current != base: + return Check( + "on base branch", + WARN, + f"HEAD is on '{current}', not the base '{base}'", + f"`git checkout {base}` if you meant to run against the base branch", + ) + return Check("on base branch", PASS, f"on '{base}'") + + +def check_leftover_task_branches(task_branches: List[str]) -> Check: + if not task_branches: + return Check("no leftover task branches", PASS, "none found") + shown = ", ".join(task_branches[:5]) + (" …" if len(task_branches) > 5 else "") + return Check( + "no leftover task branches", + WARN, + f"{len(task_branches)} leftover task/* branch(es): {shown}", + "delete them (`git branch -D `); a run uses unique names so they are safe to drop", + ) + + +def check_dangling_worktrees(worktrees: List[str]) -> Check: + if not worktrees: + return Check("no dangling worktrees", PASS, "none found") + return Check( + "no dangling worktrees", + WARN, + f"{len(worktrees)} orchestrator worktree(s) still registered", + "run `git worktree prune` (a run prunes on start, but cleaning now avoids confusion)", + ) + + +def check_models(ok: bool, detail: str) -> Check: + if ok: + return Check("models resolve", PASS, f"model preflight ok: {detail}") + return Check( + "models resolve", + FAIL, + f"model preflight failed: {detail}", + "set a valid model id in project.yaml (build.model) that your provider serves", + ) + + +def check_worktree_prime( + primed_ok: Optional[bool], healthcheck_ok: Optional[bool], detail: str = "" +) -> Check: + """``primed_ok``/``healthcheck_ok`` are True/False, or None when that step did + not apply (no setup/health command for this project).""" + if primed_ok is None and healthcheck_ok is None: + return Check( + "worktree prime + healthcheck", + PASS, + "no dependency prime needed for this project", + ) + if healthcheck_ok is False: + return Check( + "worktree prime + healthcheck", + WARN, + f"toolchain did not resolve in a throwaway worktree: {detail[:120]}", + "check the dependency install / toolchain (a run re-primes once, then flags it)", + ) + if primed_ok is False: + return Check( + "worktree prime + healthcheck", + WARN, + f"dependency prime failed: {detail[:120]}", + "verify the install command; a run falls back to the gate's own install", + ) + return Check( + "worktree prime + healthcheck", + PASS, + "a throwaway worktree primed and its toolchain resolved", + ) + + +def check_requirements(unsatisfied: List[str]) -> Check: + if not unsatisfied: + return Check("requirements satisfied", PASS, "no outstanding inputs") + shown = ", ".join(unsatisfied[:6]) + (" …" if len(unsatisfied) > 6 else "") + return Check( + "requirements satisfied", + WARN, + f"{len(unsatisfied)} REQUIREMENTS.md input(s) not yet provided: {shown}", + "answer them in .orchestrator/REQUIREMENTS.md (or set the env vars), or run with --proceed to park them", + ) diff --git a/misterdev/core/execution/early_abort.py b/misterdev/core/execution/early_abort.py deleted file mode 100644 index 74d9a65..0000000 --- a/misterdev/core/execution/early_abort.py +++ /dev/null @@ -1,98 +0,0 @@ -"""Process-signal early-abort: stop a task that is not converging. - -Today a task retries until an outcome gate fails ``max_attempts`` times or the -cost cap trips — even when the PROCESS signal already says it is stuck: the same -assertion failing attempt after attempt, or the diagnostic count refusing to -shrink. Those attempts are near-certain to fail and burn budget that a different -task (or a stronger tier, or a human) could use (docs/research-directions.md, -Theme 5 — PRM course-correction 2509.02360). - -This is a pure, deterministic monitor over per-attempt signals. It NEVER changes -correctness (a task it abandons was already failing every gate); it only decides, -early, to stop pouring attempts into a non-converging task. Two independent -triggers: - - - **stuck**: the same normalized error fingerprint recurs ``stuck_repeats`` - times in a row — the model is circling, not fixing. - - **no progress**: the diagnostic/failure count fails to strictly decrease - across a window of ``no_progress_window`` attempts — no forward motion. - -Fingerprints are normalized (paths, line numbers, hex addresses, and quoted -literals stripped) so "the same error" is recognized across attempts whose -incidental detail differs. Pure and side-effect free. -""" - -import re -from dataclasses import dataclass, field -from typing import List, Optional, Tuple - -_NORMALIZERS = ( - (re.compile(r"0x[0-9a-fA-F]+"), "0xHEX"), - (re.compile(r"\b\d+\b"), "N"), # line numbers, counts, offsets - (re.compile(r'"[^"]*"|\'[^\']*\''), "STR"), # quoted literals - (re.compile(r"(/[^\s:]+)+"), "PATH"), # file paths - (re.compile(r"\s+"), " "), -) - - -def fingerprint(error_text: str) -> str: - """A stable fingerprint of an error, ignoring incidental detail so the SAME - failure is recognized across attempts.""" - text = (error_text or "").strip().lower() - for pattern, repl in _NORMALIZERS: - text = pattern.sub(repl, text) - return text.strip()[:400] - - -@dataclass -class AttemptSignal: - """One attempt's process signal: how many diagnostics, and their fingerprint.""" - - error_count: int - error_text: str = "" - - @property - def fingerprint(self) -> str: - return fingerprint(self.error_text) - - -@dataclass -class ConvergenceMonitor: - """Accumulates per-attempt signals and reports when to abort early. - - ``stuck_repeats`` and ``no_progress_window`` count ATTEMPTS, not calls; both - require at least that many signals before they can fire, so a monitor never - aborts before it has evidence. Disabled triggers: set either to 0. - """ - - stuck_repeats: int = 3 - no_progress_window: int = 3 - _signals: List[AttemptSignal] = field(default_factory=list) - - def update(self, signal: AttemptSignal) -> None: - self._signals.append(signal) - - def _stuck(self) -> bool: - if self.stuck_repeats < 2 or len(self._signals) < self.stuck_repeats: - return False - recent = self._signals[-self.stuck_repeats :] - first = recent[0].fingerprint - return bool(first) and all(s.fingerprint == first for s in recent) - - def _no_progress(self) -> bool: - if self.no_progress_window < 2 or len(self._signals) < self.no_progress_window: - return False - counts = [s.error_count for s in self._signals[-self.no_progress_window :]] - # No strict decrease anywhere in the window -> not making forward motion. - return all(b >= a for a, b in zip(counts, counts[1:])) - - def should_abort(self) -> Tuple[bool, Optional[str]]: - """``(abort, reason)`` — advisory; the caller decides what to do (escalate - tier, backtrack, or stop). A task that has never had a signal never aborts.""" - if self._stuck(): - fp = self._signals[-1].fingerprint - return True, f"stuck: same error {self.stuck_repeats}x in a row ({fp[:80]})" - if self._no_progress(): - counts = [s.error_count for s in self._signals[-self.no_progress_window :]] - return True, f"no progress: diagnostics not shrinking over {counts}" - return False, None diff --git a/misterdev/core/execution/env_learnings.py b/misterdev/core/execution/env_learnings.py new file mode 100644 index 0000000..e24b9a1 --- /dev/null +++ b/misterdev/core/execution/env_learnings.py @@ -0,0 +1,109 @@ +"""Per-project environment memory that persists across runs. + +Every run otherwise re-discovers the same durable facts about a project's +environment: which dependency-prime command actually works, how many parallel +workers the machine sustains before it thrashes (see ``adaptive.py``), which +toolchain-resolve probe is right, and any hard ordering constraint learned the +hard way. This file records those facts under ``.orchestrator/env_learnings.json`` +so the NEXT run starts pre-tuned instead of relearning from scratch — the +cross-run self-improvement loop. + +Schema (tiny and forward-compatible; every field optional): + + { + "version": 1, + "worktree_setup_command": "pnpm install --prefer-offline" | null, + "worktree_healthcheck_command": "npx --no-install tsc --version" | null, + "max_workers": 4 | null, // a learned REDUCTION from backoff + "ordering_constraints": ["dashboard build must precede server test"] + } + +Applying a learned value NEVER overrides an explicit ``project.yaml`` setting — +the user's config always wins; a learning only fills a gap the user left open. +""" + +import json +from dataclasses import dataclass, field +from typing import List, Optional + +from misterdev.logging_setup import setup_logger +from misterdev.utils.file_utils import atomic_write, orchestrator_state_file + +logger = setup_logger(__name__) + +_VERSION = 1 +_FILE = "env_learnings.json" + +# (learning attribute, config section, config key). The only fields that pre-tune +# a runtime SETTING; ordering_constraints is persisted advisory data, not a knob. +_TUNABLES = ( + ("worktree_setup_command", "orchestrator", "worktree_setup_command"), + ("worktree_healthcheck_command", "orchestrator", "worktree_healthcheck_command"), + ("max_workers", "orchestrator", "max_workers"), +) + + +@dataclass +class EnvLearnings: + """Durable, per-project environment facts learned across runs.""" + + worktree_setup_command: Optional[str] = None + worktree_healthcheck_command: Optional[str] = None + max_workers: Optional[int] = None + ordering_constraints: List[str] = field(default_factory=list) + + @classmethod + def load(cls, project_path) -> "EnvLearnings": + """Load the project's learnings, or an empty set. Never raises: a missing + or corrupt file self-heals to empty so a bad ledger can't break a run.""" + path = orchestrator_state_file(project_path, _FILE) + if not path.exists(): + return cls() + try: + data = json.loads(path.read_text(encoding="utf-8")) + except (json.JSONDecodeError, OSError, ValueError) as e: + logger.warning(f"Env-learnings unreadable ({e}); starting empty.") + return cls() + if not isinstance(data, dict): + return cls() + mw = data.get("max_workers") + oc = data.get("ordering_constraints") + return cls( + worktree_setup_command=data.get("worktree_setup_command") or None, + worktree_healthcheck_command=data.get("worktree_healthcheck_command") + or None, + max_workers=int(mw) if isinstance(mw, int) and mw > 0 else None, + ordering_constraints=[str(c) for c in oc] if isinstance(oc, list) else [], + ) + + def save(self, project_path) -> None: + """Persist atomically under ``.orchestrator/``. Best-effort.""" + path = orchestrator_state_file(project_path, _FILE) + data = { + "version": _VERSION, + "worktree_setup_command": self.worktree_setup_command, + "worktree_healthcheck_command": self.worktree_healthcheck_command, + "max_workers": self.max_workers, + "ordering_constraints": list(self.ordering_constraints), + } + atomic_write(path, json.dumps(data, indent=2)) + + def apply_to_config(self, config: dict) -> List[str]: + """Pre-tune ``config`` from learned values WITHOUT overriding explicit ones. + + For each tunable, a learned value is written into the config only when the + user left that key unset (an explicit ``project.yaml`` value always wins). + Returns a human-readable list of what was applied, for logging. Mutates the + config dict in place (the same dict ``get_setting`` reads). + """ + applied: List[str] = [] + for attr, section, key in _TUNABLES: + value = getattr(self, attr) + if value is None: + continue + sect = config.get(section) + if isinstance(sect, dict) and key in sect: + continue # explicit config wins — never override it + config.setdefault(section, {})[key] = value + applied.append(f"{section}.{key}={value!r}") + return applied diff --git a/misterdev/core/execution/error_classifier.py b/misterdev/core/execution/error_classifier.py index 5e3d605..9cb1786 100644 --- a/misterdev/core/execution/error_classifier.py +++ b/misterdev/core/execution/error_classifier.py @@ -315,9 +315,11 @@ def classify_error(error_output: str) -> str: for code, category in _RUST_ERROR_CODES.items(): if f"error[{code}]" in error_output: return category - # Fast path: C#/Roslyn error codes (e.g. "error CS0103:") + # Fast path: C#/Roslyn error codes. Anchor on the Roslyn prefix `error CSxxxx:` + # (emitted as `(line,col): error CSxxxx:`) — a bare `CSxxxx:` substring in + # prose, a doc URL, or an assertion log must NOT hijack the classification. for code, category in _CSHARP_ERROR_CODES.items(): - if f"{code}:" in error_output: + if f"error {code}:" in error_output: return category lower = error_output.lower() diff --git a/misterdev/core/execution/error_resolver.py b/misterdev/core/execution/error_resolver.py index 7ce88bd..83aa9fe 100644 --- a/misterdev/core/execution/error_resolver.py +++ b/misterdev/core/execution/error_resolver.py @@ -31,6 +31,23 @@ re.compile(r"at (?P[^\s(]+)\(.*:(?P\d+)\)", re.MULTILINE), ] +# Patterns that capture the NAME a diagnostic refers to (not the error location). +# Broad by design: only names that resolve to a real project symbol are surfaced, +# so a loose match on a compiler primitive (`u32`) simply finds nothing. +_REFERENCED_NAME_PATTERNS: list[re.Pattern] = [ + # rustc: cannot find function/value/type/trait/... `name` + re.compile( + r"cannot find (?:value|function|type|trait|macro|struct|enum|variant|" + r"method|attribute|import|module) `(?P[\w:]+)`" + ), + # tsc: Cannot find name 'Name' + re.compile(r"[Cc]annot find name ['\"](?P[\w$]+)['\"]"), + # type mismatch, either compiler: expected `A`, found `B` + re.compile(r"(?:expected|found) `(?P[\w:]+)`"), + # tsc: Type 'A' is not assignable to type 'B' + re.compile(r"[Tt]ype ['\"](?P[\w$]+)['\"]"), +] + class ErrorLocation: """A single attributed error location.""" @@ -114,21 +131,35 @@ def resolve_errors(self, error_output: str) -> List[ErrorLocation]: return locations - def format_for_llm(self, locations: List[ErrorLocation]) -> str: - """Format attributed locations as a prompt-ready string.""" - if not locations: - return "" - - lines = ["## Error Attribution"] - for loc in locations[:10]: # cap to avoid bloating context - lines.append(f"\n### {loc.file}:{loc.line}") - if loc.symbol: - lines.append(f"- Symbol: `{loc.symbol}`") - callers = self._callers(loc.symbol_key) - if callers: - lines.append(f"- Called by: {', '.join(callers)}") - if loc.snippet: - lines.append(f"```\n{loc.snippet}\n```") + def format_for_llm( + self, locations: List[ErrorLocation], error_output: str = "" + ) -> str: + """Format attributed locations as a prompt-ready string. + + When ``error_output`` is supplied and a symbol graph is available, any + symbol NAMED by a diagnostic ("cannot find X", "expected `A`, found `B`", + tsc "Cannot find name 'X'") is looked up and its DEFINITION is surfaced — + so a "cannot find function `helper`" leads the model to where ``helper`` + actually lives (a typo, a wrong import) instead of only the error line's + enclosing symbol. + """ + lines: List[str] = [] + if locations: + lines.append("## Error Attribution") + for loc in locations[:10]: # cap to avoid bloating context + lines.append(f"\n### {loc.file}:{loc.line}") + if loc.symbol: + lines.append(f"- Symbol: `{loc.symbol}`") + callers = self._callers(loc.symbol_key) + if callers: + lines.append(f"- Called by: {', '.join(callers)}") + if loc.snippet: + lines.append(f"```\n{loc.snippet}\n```") + + referenced = self._referenced_definitions(error_output) + if referenced: + lines.append("\n## Referenced symbol definitions") + lines.extend(referenced) return "\n".join(lines) @@ -136,6 +167,51 @@ def format_for_llm(self, locations: List[ErrorLocation]) -> str: # Internal helpers # ------------------------------------------------------------------ + def _referenced_definitions( + self, error_output: str, max_defs: int = 5, max_lines: int = 8 + ) -> List[str]: + """Rendered definition blocks for the symbols a diagnostic names. + + Extracts the referenced names, resolves each to a project symbol by name, + and renders its declaration (kind, file:line, body). Empty when there is no + graph, no diagnostic name, or no name resolves to a known symbol. + """ + if not error_output: + return [] + symbols = getattr(self.graph, "symbols", None) + if not symbols: + return [] + names: List[str] = [] + for pattern in _REFERENCED_NAME_PATTERNS: + for match in pattern.finditer(error_output): + leaf = match.group("name").split("::")[-1].split(".")[-1] + if leaf and leaf not in names: + names.append(leaf) + if not names: + return [] + by_name: dict[str, List[Any]] = {} + for node in symbols.values(): + by_name.setdefault(node.name, []).append(node) + blocks: List[str] = [] + seen: set[tuple] = set() + for name in names: + for node in by_name.get(name, []): + key = (node.file_path, node.name, node.start_line) + if key in seen: + continue + seen.add(key) + body = "\n".join((node.content or "").splitlines()[:max_lines]) + block = ( + f"\n### `{node.name}` ({node.kind}) — " + f"{node.file_path}:{node.start_line + 1}" + ) + if body: + block += f"\n```\n{body}\n```" + blocks.append(block) + if len(blocks) >= max_defs: + return blocks + return blocks + def _read_snippet(self, file_str: str, line_no: int, context: int = 5) -> str: """Return a few lines of source around *line_no* for context.""" # Try relative to project root first, then as absolute path. diff --git a/misterdev/core/execution/escalation.py b/misterdev/core/execution/escalation.py new file mode 100644 index 0000000..3cb607e --- /dev/null +++ b/misterdev/core/execution/escalation.py @@ -0,0 +1,68 @@ +"""Escalation ladder for a task that keeps failing the same way. + +A task retried identically tends to fail identically. This ladder escalates the +RESPONSE as a task accumulates NON-INFRA (real code) failures, in order: + + normal -> widen_context -> full_rewrite -> stronger_model -> decompose + +- ``widen_context``: show the full target file(s), their dependents, and the + task's pre-decided spec/acceptance verbatim — the model was probably editing + blind or against a truncated view. +- ``full_rewrite``: a STRUCTURALLY different attempt — stop patching (SEARCH/ + REPLACE against a view the model keeps mis-anchoring on) and rewrite the whole + target region from scratch. This is not another reprompt of the same approach; + it changes HOW the edit is produced while keeping the same task and model. +- ``stronger_model``: the cheap/default model can't crack it; route the + generation to a more capable one. +- ``decompose``: the task is too big to land in one edit; request a split into + named sub-steps instead of failing outright. + +Crucially, an ENVIRONMENT fault (a timeout, a locked store — see ``infra.py``) +self-heals elsewhere and is NOT the code's fault, so it must never advance the +ladder. ``should_count_failure`` is the gate: only failures it counts move the +rung. The rung choice itself is a pure function of that count, so the whole +policy is unit-testable in isolation. +""" + +from misterdev.core.execution.infra import infra_failure + +# In escalating order; index doubles as the rung's strength. +RUNGS = ("normal", "widen_context", "full_rewrite", "stronger_model", "decompose") + + +def should_count_failure(output: str) -> bool: + """True when a gate failure is a real CODE failure that advances the ladder. + + An environment/infra fault (timeout, locked store, OOM) self-heals and is not + the code's doing, so it returns False and the ladder holds where it is. + """ + return infra_failure(output) is None + + +def choose_rung( + code_failures: int, + *, + widen_after: int = 1, + rewrite_after: int = 2, + model_after: int = 3, + decompose_after: int = 3, +) -> str: + """The escalation rung for the NEXT attempt, given prior NON-infra failures. + + ``code_failures`` is the number of real code failures the task has already + taken (0 on the first attempt -> ``normal``). Highest threshold that is met + wins, so misordered thresholds still yield a defined (monotonic) rung. The + default ``model_after``/``decompose_after`` coincide at 3 so that within the + default 3-attempt budget a task climbs normal -> widen -> full_rewrite and then + decomposes at exhaustion; ``stronger_model`` (a no-op unless ``escalation_model`` + is set) becomes reachable only when ``decompose_after`` is raised. + """ + if code_failures >= decompose_after: + return "decompose" + if code_failures >= model_after: + return "stronger_model" + if code_failures >= rewrite_after: + return "full_rewrite" + if code_failures >= widen_after: + return "widen_context" + return "normal" diff --git a/misterdev/core/execution/failure_taxonomy.py b/misterdev/core/execution/failure_taxonomy.py new file mode 100644 index 0000000..6011eba --- /dev/null +++ b/misterdev/core/execution/failure_taxonomy.py @@ -0,0 +1,116 @@ +"""Classify a run's terminal non-successes so failures become signal, not noise. + +At the end of a run the interesting question is "WHY did it underperform?" — was +the box thrashing (infra), was a credential missing (blocked), did tasks collide +on merge, did the code just not work, or did tasks park for input? This module +routes each terminal non-success into one taxonomy bucket and aggregates a +one-glance run summary. It reuses the existing pure signal detectors +(``infra_failure``, ``blocked_reason``) so a fault is labelled the same way here +as it was handled during the run. + +Pure and side-effect-free: classification is a function of (status, text), and +the summary is a function of the per-task classifications — so both the routing +and the aggregation are directly unit-testable. The orchestrator supplies the +texts and writes the result to the console and ``.orchestrator/run_summary.json``. +""" + +import re +from typing import Dict, List, Optional, Tuple + +from misterdev.core.execution.blocker import blocked_reason +from misterdev.core.execution.infra import infra_failure + +# Ordered most-specific → most-generic; also the tie-break order for the top +# obstacle (an earlier, more-specific category wins an equal count). +CATEGORIES: Tuple[str, ...] = ( + "infra", + "blocked-external", + "merge-conflict", + "acceptance-unmet", + "genuine-code-failure", + "deferred-needs-input", +) + +_MERGE_CONFLICT = re.compile( + r"merge conflict|merge failed|CONFLICT \(content\)|post-merge health gate", re.I +) +_ACCEPTANCE = re.compile(r"acceptance crit(?:erion|eria).{0,40}not met", re.I) + + +def classify_failure(status: str, text: str) -> str: + """Route one terminal non-success into a taxonomy category. + + Strongest signal wins, independent of ``status``: an environment or blocked + fault is labelled as such even on a task recorded "failed", because that IS + why it failed. Only when no signal matches does ``status`` decide — a parked + task is ``deferred-needs-input``, anything else ``genuine-code-failure``. + """ + t = text or "" + if blocked_reason(t): + return "blocked-external" + if infra_failure(t): + return "infra" + if _MERGE_CONFLICT.search(t): + return "merge-conflict" + if _ACCEPTANCE.search(t): + return "acceptance-unmet" + if (status or "").strip().lower() == "deferred": + return "deferred-needs-input" + return "genuine-code-failure" + + +def _first_meaningful_line(text: str) -> str: + """The first informative line of a failure blob — skipping blanks and markdown + fences/headers that would otherwise make a useless exemplar.""" + for line in (text or "").splitlines(): + s = line.strip() + if s and not s.startswith("```") and not s.startswith("#"): + return s + return "" + + +def build_run_summary( + completed: int, + failed_items: List[Tuple[str, str]], + deferred_items: List[Tuple[str, str]], + elapsed_seconds: float, +) -> Dict: + """Aggregate a run's outcomes into a one-glance summary. + + ``failed_items``/``deferred_items`` are ``(task_id, failure_text)`` pairs. + Returns counts, the failure breakdown by category (zero categories omitted), a + short exemplar message per category, wall-clock, and the single top recurring + obstacle (the category with the most failures; ties break toward the more + specific one). Deterministic given the same inputs. + """ + breakdown: Dict[str, int] = {c: 0 for c in CATEGORIES} + exemplars: Dict[str, str] = {} + + def note(status: str, text: str) -> None: + cat = classify_failure(status, text) + breakdown[cat] += 1 + if cat not in exemplars: + msg = _first_meaningful_line(text) + if msg: + exemplars[cat] = msg[:200] + + for _id, text in failed_items: + note("failed", text) + for _id, text in deferred_items: + note("deferred", text) + + nonzero = {c: n for c, n in breakdown.items() if n} + top_obstacle: Optional[str] = None + if nonzero: + # Most failures wins; a tie breaks toward the earlier (more specific) + # category via its negative index. + top_obstacle = max(nonzero, key=lambda c: (nonzero[c], -CATEGORIES.index(c))) + return { + "completed": completed, + "failed": len(failed_items), + "deferred": len(deferred_items), + "elapsed_seconds": round(float(elapsed_seconds), 1), + "failure_breakdown": nonzero, + "exemplars": exemplars, + "top_obstacle": top_obstacle, + } diff --git a/misterdev/core/execution/infra.py b/misterdev/core/execution/infra.py new file mode 100644 index 0000000..388114f --- /dev/null +++ b/misterdev/core/execution/infra.py @@ -0,0 +1,70 @@ +"""Classify a gate failure as an ENVIRONMENT/infrastructure fault, not a code bug. + +Some gate failures say nothing about the code under test: a command timed out, a +dependency was never installed, the package store was locked by a concurrent +install, the disk filled, the box ran out of memory/file handles. Reflecting on +these as if they were code defects wastes a model call and can revert correct +work; the right response is to repair the environment and re-run, not to edit. + +This is the counterpart to ``blocker.py``: ``blocker`` detects an EXTERNAL +resource the user must supply (park the task); ``infra`` detects a TRANSIENT +local fault the orchestrator should self-heal by retrying. Both are pure, +signal-only, and conservative — a plain assertion or type error must never be +misread as infrastructure. +""" + +import re +from typing import Optional, Tuple + +# (compiled signal, human reason). Every phrase is one an ENVIRONMENT fault emits, +# never something a normal type/assertion failure prints, so a real code error is +# not misclassified as infra (at worst a false positive costs one extra re-run). +_SIGNALS: Tuple[Tuple[re.Pattern, str], ...] = ( + ( + re.compile(r"timed out after \d+s|\bETIMEDOUT\b|operation timed out", re.I), + "a command timed out", + ), + ( + re.compile( + r"cannot find module|\bMODULE_NOT_FOUND\b|ERR_MODULE_NOT_FOUND|" + r"is not the tsc command|\bcommand not found\b|" + r"no such file or directory, open .*node_modules", + re.I, + ), + "a dependency was not installed in the worktree", + ), + ( + re.compile( + r"\bELOCK\b|store is locked|waiting for the lock|" + r"another process is running|lock file .* held", + re.I, + ), + "the package store was locked by a concurrent install", + ), + ( + re.compile(r"\bENOSPC\b|no space left on device", re.I), + "the disk is full", + ), + ( + re.compile(r"\bEMFILE\b|\bENFILE\b|too many open files", re.I), + "the process ran out of file handles", + ), + ( + re.compile( + r"out of memory|JavaScript heap out of memory|\bSIGKILL\b|" + r"\bENOMEM\b|Killed\s*$", + re.I, + ), + "the process ran out of memory or was killed", + ), +) + + +def infra_failure(output: str) -> Optional[str]: + """A short human reason when ``output`` shows an environment/infra fault, else + None. None means "treat as an ordinary (code) failure".""" + text = output or "" + for pattern, reason in _SIGNALS: + if pattern.search(text): + return reason + return None diff --git a/misterdev/core/execution/integration_gate.py b/misterdev/core/execution/integration_gate.py new file mode 100644 index 0000000..4270926 --- /dev/null +++ b/misterdev/core/execution/integration_gate.py @@ -0,0 +1,416 @@ +"""Post-wave integration gate + regression revert, extracted from the orchestrator. + +``ProjectOrchestrator`` mixes this in: every method uses only ``self`` and the +``executor`` passed to it, so the split is behavior-preserving. The unit owns the +"did merging this wave regress the suite, and if so revert the culprits" concern: +the full-suite gate (binary / count / identity modes for a red baseline), the +per-target polyglot analogue, the shared failure-count / failing-id parsers, and +the post-build bisect-and-revert. +""" + +from typing import Optional + +from misterdev.core.execution.project import Project +from misterdev.core.models import Task +from misterdev.core.modes import BuildFlags +from misterdev.core.planning.assessment import ProjectAssessment +from misterdev.core.reporting.report import BuildReport +from misterdev.logging_setup import setup_logger +from misterdev.task_executors.markdown_plan_executor import MarkdownPlanExecutor + +logger = setup_logger(__name__) + + +class IntegrationGateMixin: + @staticmethod + def _wave_commits(executor, project, tasks) -> list: + """Collect ``(task_id, sha)`` for each task that has a recorded commit, + skipping tasks with none. Shared by the regression-revert and + integration-gate paths.""" + commits = [] + for t in tasks: + sha = executor.find_task_commit(project, t.id) + if sha: + commits.append((t.id, sha)) + return commits + + def _maybe_rollback_regression( + self, + project: Project, + report: BuildReport, + assessment: ProjectAssessment, + flags: BuildFlags, + ) -> None: + """If the post-build gate failed, bisect task commits and revert the culprit.""" + if flags.no_rollback or not report.completed_tasks: + return + test_cmd = assessment.structure.test_command + if not test_cmd: + return + ex = MarkdownPlanExecutor() + if not ex._is_git_repo(project): + return + commits = self._wave_commits(ex, project, report.completed_tasks) + if not commits: + return + logger.warning("Post-build regression detected; bisecting task commits...") + culprit = ex.bisect_regression(project, commits, test_cmd) + if not culprit: + logger.info("Bisect did not isolate a single task commit.") + return + sha = dict(commits)[culprit] + if ex.revert_task_commit(project, sha): + logger.warning( + f"Regression bisected to {culprit}; commit {sha[:8]} reverted." + ) + report.key_decisions.append( + f"Regression from {culprit} auto-reverted (bisect)" + ) + + def _suite_failures( + self, + project: Project, + executor: MarkdownPlanExecutor, + test_cmd: str, + timeout: int, + cwd=None, + ) -> Optional[int]: + """Full-suite failure count: 0 when green, the parsed count when red, or + None when the count can't be parsed (caller then can't count-compare). + + ``cwd`` runs the command in a sub-project (target) directory; defaults to + the repo root.""" + from misterdev.core.verification.validator import ( + _parse_test_counts, + ) + + ok, output = executor._run_command(project, test_cmd, timeout=timeout, cwd=cwd) + if ok: + return 0 + total, failures = _parse_test_counts(output) + return failures if total > 0 else None + + @staticmethod + def _failing_ids_from_output(output: str, project: Project) -> Optional[set]: + """The SET of failing test identifiers parsed from runner output, or None + when none can be parsed (caller falls back to the count). + + Identity beats a bare count: it lets the integration gate revert a wave + that offsets a genuine fix against a NEW break (net-zero count, which + count mode waves through) and stays correct if a fix renames/reorders + tests. Reuses the FailureView parsers already validated per runner.""" + from misterdev.core.execution.failure_view import extract_failures + + lang = ( + (project.config.get("language") or "") + if getattr(project, "config", None) + else "" + ) + ids = { + f.test + for f in extract_failures(output, language=lang) + if getattr(f, "test", "") + } + return ids or None + + def _suite_failing_ids( + self, + project: Project, + executor: MarkdownPlanExecutor, + test_cmd: str, + timeout: int, + cwd=None, + ) -> Optional[set]: + """Full-suite failing-test id SET: empty when green, the parsed ids when + red, or None when unparseable (caller falls back to the count).""" + ok, output = executor._run_command(project, test_cmd, timeout=timeout, cwd=cwd) + if ok: + return set() + return self._failing_ids_from_output(output, project) + + @staticmethod + def _looks_like_broken_build(output: str) -> bool: + """True when a FAILING suite's output shows a structural break — the code no + longer compiles, collects, or imports — rather than ordinary (countable) + test failures we merely could not parse. A build break is strictly worse + than any counted red baseline (the suite no longer even runs), so it must be + reverted even though it yields no count; an unrecognized-but-ran failure + stays ambiguous and is left alone.""" + from misterdev.core.execution.compile_view import extract_compile_errors + + if extract_compile_errors(output): + return True + low = (output or "").lower() + signals = ( + "error collecting", + "errors during collection", + "importerror", + "modulenotfounderror", + "no module named", + "syntaxerror", + "internalerror", + "cannot find module", + "cannot import", + "unresolved import", + "segmentation fault", + ) + return any(s in low for s in signals) + + def _suite_broken( + self, project, executor, test_cmd: str, timeout: int, cwd=None + ) -> bool: + """True when the suite command fails AND its output is a build/collection + break. Runs the command once; used only on the unparseable path to tell a + genuine break (revert) from an ambiguous unrecognized failure (leave).""" + ok, output = executor._run_command(project, test_cmd, timeout=timeout, cwd=cwd) + return (not ok) and self._looks_like_broken_build(output) + + def _integration_gate_count( + self, + project: Project, + executor: MarkdownPlanExecutor, + test_cmd: str, + wave_tasks: list[Task], + timeout: int, + baseline_failures: int, + ) -> list[str]: + """Count-mode gate for a RED baseline: revert wave commits (newest first) + only when the wave RAISED the full-suite failure count above the baseline. + + This closes the gap where, with the binary gate disabled by a red + baseline, a task gated on its own scoped tests could worsen the overall + suite and still commit. An unparseable post-wave count is left alone (we + do not revert on a number we can't read). + """ + after = self._suite_failures(project, executor, test_cmd, timeout) + if after is None: + if not self._suite_broken(project, executor, test_cmd, timeout): + return [] + logger.warning( + "Integration gate (count): post-wave suite no longer builds/collects " + "(strictly worse than the red baseline); reverting wave commits." + ) + elif after <= baseline_failures: + return [] + else: + logger.warning( + f"Integration gate (count): failures rose {baseline_failures} -> " + f"{after}; reverting wave commits until restored." + ) + commits = self._wave_commits(executor, project, wave_tasks) + reverted: list[str] = [] + for tid, sha in reversed(commits): + if executor.revert_task_commit(project, sha): + reverted.append(tid) + now = self._suite_failures(project, executor, test_cmd, timeout) + if now is not None and now <= baseline_failures: + break + return reverted + + def _integration_gate_ids( + self, + project: Project, + executor: MarkdownPlanExecutor, + test_cmd: str, + wave_tasks: list[Task], + timeout: int, + baseline_ids: set, + ) -> list[str]: + """Identity-mode gate for a RED baseline: revert the wave iff it introduced + a NEW failing test (one not failing at baseline), regardless of the failure + COUNT. + + Stricter and more correct than count mode: a wave that fixes test A but + breaks test B nets zero on the count and slips past ``_integration_gate_count``, + yet it introduced a real regression (B) — identity mode reverts it. A wave + that resolves none of the baseline failures and adds none (a no-op "fix" + that still merged) is surfaced as no-progress rather than silently blessed. + """ + after = self._suite_failing_ids(project, executor, test_cmd, timeout) + if after is None: + if not self._suite_broken(project, executor, test_cmd, timeout): + return [] # unparseable-but-ran; don't revert on ids we can't read + logger.warning( + "Integration gate (identity): post-wave suite no longer builds/" + "collects; reverting wave commits until restored." + ) + commits = self._wave_commits(executor, project, wave_tasks) + reverted: list[str] = [] + for tid, sha in reversed(commits): + if executor.revert_task_commit(project, sha): + reverted.append(tid) + now = self._suite_failing_ids(project, executor, test_cmd, timeout) + if now is not None and not (now - baseline_ids): + break + return reverted + new_failures = after - baseline_ids + if not new_failures: + if baseline_ids - after: + logger.info( + "Integration gate (identity): resolved " + f"{len(baseline_ids - after)} baseline failure(s), no regressions." + ) + else: + logger.info( + "Integration gate (identity): wave added no new failures but " + "resolved none either — no progress on the failing suite." + ) + return [] + logger.warning( + f"Integration gate (identity): {len(new_failures)} new failing test(s) " + f"(e.g. {sorted(new_failures)[:2]}); reverting wave commits until restored." + ) + commits = self._wave_commits(executor, project, wave_tasks) + reverted: list[str] = [] + for tid, sha in reversed(commits): + if executor.revert_task_commit(project, sha): + reverted.append(tid) + now = self._suite_failing_ids(project, executor, test_cmd, timeout) + if now is not None and not (now - baseline_ids): + break + return reverted + + @staticmethod + def _target_regressed(after: Optional[int], baseline: Optional[int]) -> bool: + """Did a target's gate regress vs its baseline? + + ``after``/``baseline`` are :meth:`_suite_failures` results (0 green, N + count, None unparseable). A green-now gate never regressed. With no + countable baseline we can't compare, so we don't revert. A binary failure + now (None) is a regression only if the target was green (baseline 0); + otherwise compare counts. + """ + if after == 0: + return False + if baseline is None: + return False + if after is None: + return baseline == 0 + return after > baseline + + def _integration_gate_targets( + self, + project: Project, + executor: MarkdownPlanExecutor, + targets: list[dict], + wave_tasks: list[Task], + timeout: int, + target_baselines: dict, + ) -> list[str]: + """Per-target integration gate: validate each sub-project the wave touched + with ITS own toolchain (in ITS directory), reverting only the wave commits + belonging to a target that regressed. This is the multi-target analogue of + :meth:`_integration_gate` — the last place a polyglot run would otherwise + gate with the wrong toolchain. + """ + from misterdev.core.planning.targets import select_target + + reverted: list[str] = [] + for tgt in targets: + gate_cmd = tgt.get("test_command") or tgt.get("build_command") + if not gate_cmd: + continue + tname = tgt.get("name") or tgt.get("path") + tp = (tgt.get("path") or "").strip("/") + run_dir = project.path / tp if tp else project.path + owned = [ + t + for t in wave_tasks + if ( + select_target( + targets, list(t.files_to_modify) + list(t.files_to_create) + ) + or {} + ).get("path") + == tgt.get("path") + ] + if not owned: + continue + baseline = target_baselines.get(tname) + after = self._suite_failures( + project, executor, gate_cmd, timeout, cwd=run_dir + ) + if not self._target_regressed(after, baseline): + continue + logger.warning( + f"Integration gate [{tname}]: regressed (baseline={baseline}, " + f"after={after}); reverting this target's wave commits." + ) + commits = [(t.id, executor.find_task_commit(project, t.id)) for t in owned] + commits = [(tid, sha) for tid, sha in commits if sha] + for tid, sha in reversed(commits): + if executor.revert_task_commit(project, sha): + reverted.append(tid) + now = self._suite_failures( + project, executor, gate_cmd, timeout, cwd=run_dir + ) + if not self._target_regressed(now, baseline): + break + return reverted + + def _integration_gate( + self, + project: Project, + executor: MarkdownPlanExecutor, + test_cmd: str, + wave_tasks: list[Task], + timeout: int, + baseline_failures: int = 0, + ) -> list[str]: + """Run the full suite after a wave; revert task commits that regressed it. + + Returns the task_ids whose commits were reverted (empty if the suite + still passes). Bisects to the single culprit when possible; if that + can't isolate it or the tree is still red afterward, reverts the + remaining wave commits (newest first) to restore a green baseline. On a + RED baseline it prefers IDENTITY mode (revert a wave that adds any new + failing test, so an offsetting fix/break can't slip through) when the + baseline's failing set was parseable, falling back to COUNT mode (revert + only a wave that raises the failure count) otherwise. + """ + # Prefer identity mode (revert on any NEW failing test) over count mode + # (revert only when the count rises) whenever the baseline's failing set + # was parseable: it also catches an offsetting fix/break that count mode + # nets to zero. Count mode remains the fallback for unparseable output. + baseline_ids = getattr(project, "baseline_test_failing_ids", None) + if baseline_ids: + return self._integration_gate_ids( + project, executor, test_cmd, wave_tasks, timeout, baseline_ids + ) + if baseline_failures > 0: + return self._integration_gate_count( + project, executor, test_cmd, wave_tasks, timeout, baseline_failures + ) + ok, _ = executor._run_command(project, test_cmd, timeout=timeout) + if ok: + return [] + + commits = self._wave_commits(executor, project, wave_tasks) + if not commits: + logger.warning( + "Integration gate: suite regressed but no task commits found to revert." + ) + return [] + + logger.warning("Integration gate: suite regressed; isolating culprit...") + reverted: list[str] = [] + culprit = executor.bisect_regression( + project, commits, test_cmd, timeout=timeout + ) + if culprit: + sha = dict(commits)[culprit] + if executor.revert_task_commit(project, sha): + reverted.append(culprit) + ok, _ = executor._run_command(project, test_cmd, timeout=timeout) + if ok: + return reverted + + for tid, sha in reversed(commits): + if tid in reverted: + continue + if executor.revert_task_commit(project, sha): + reverted.append(tid) + ok, _ = executor._run_command(project, test_cmd, timeout=timeout) + if ok: + break + return reverted diff --git a/misterdev/core/execution/jobs.py b/misterdev/core/execution/jobs.py new file mode 100644 index 0000000..18a3c25 --- /dev/null +++ b/misterdev/core/execution/jobs.py @@ -0,0 +1,347 @@ +"""In-process registry for long-running build/run jobs. + +MCP tools are synchronous: a ``build`` call blocks the client (and the server's +event loop) until the whole autonomous run finishes — minutes for a real build, +long enough to trip client timeouts, with no way to monitor or stop it. This +registry runs a build/run in a background daemon thread and hands back a +``run_id`` immediately, so a client can poll ``status`` and request ``stop``. + +Deliberately NOT a port of the donor's job machinery, whose concurrency was the +source of its critical bugs. Two guards address those failure modes directly: + +- **One job per project** (donor C-010/CLU-003: unguarded concurrent runs on the + same directory corrupted files/state). ``start`` refuses to launch a second + job for a path that already has one running. +- **No silent thread death** (donor C-007: double-resolve / unhandled rejection). + The runner captures every exception and records it on the job, so a crashed + build becomes an observable ``failed`` status, never a lost thread. + +Stop is cooperative and reuses misterdev's existing graceful kill-switch rather +than interrupting the task loop: the ``stop_hook`` lowers the run's budget so the +next model call raises ``BudgetExceededError``, which the orchestrator already +catches and degrades to a partial report. In-flight work finishes; no new work +starts. +""" + +from __future__ import annotations + +import threading +import uuid +from dataclasses import dataclass, field +from datetime import datetime, timezone +from pathlib import Path +from typing import Any, Callable, Dict, List, Optional + +from misterdev.logging_setup import setup_logger + +logger = setup_logger(__name__) + + +def _now() -> str: + return datetime.now(timezone.utc).isoformat() + + +def _same_path(a: str, b: str) -> bool: + """True when two paths point at the same directory (resolved).""" + try: + return Path(a).expanduser().resolve() == Path(b).expanduser().resolve() + except OSError: + return a == b + + +@dataclass +class Job: + """A single background build/run and its observable state.""" + + run_id: str + kind: str # "build" | "run" + project_path: str + status: str = "running" # running | succeeded | failed | stopped + result: Optional[str] = None + error: Optional[str] = None + started_at: str = field(default_factory=_now) + ended_at: Optional[str] = None + stop_requested: bool = False + # Task-level progress the running build reports (0/0 until it does). + tasks_done: int = 0 + tasks_total: int = 0 + phase: str = "" + message: str = "" + _stop_hook: Optional[Callable[[], None]] = None + _thread: Optional[threading.Thread] = None + + def to_dict(self) -> Dict[str, Any]: + return { + "run_id": self.run_id, + "kind": self.kind, + "project_path": self.project_path, + "status": self.status, + "result": self.result, + "error": self.error, + "started_at": self.started_at, + "ended_at": self.ended_at, + "stop_requested": self.stop_requested, + "tasks_done": self.tasks_done, + "tasks_total": self.tasks_total, + "phase": self.phase, + "message": self.message, + } + + @classmethod + def from_dict(cls, d: Dict[str, Any]) -> "Job": + """Reconstruct a job from its persisted form (no live thread/hook). + + A job persisted as ``running`` belonged to a process that has since died, + so its thread no longer exists: load it as ``interrupted`` rather than + implying it is still making progress in this process.""" + status = d.get("status", "interrupted") + if status == "running": + status = "interrupted" + return cls( + run_id=str(d.get("run_id", "")), + kind=str(d.get("kind", "")), + project_path=str(d.get("project_path", "")), + status=status, + result=d.get("result"), + error=d.get("error"), + started_at=str(d.get("started_at", _now())), + ended_at=d.get("ended_at"), + stop_requested=bool(d.get("stop_requested", False)), + tasks_done=int(d.get("tasks_done", 0) or 0), + tasks_total=int(d.get("tasks_total", 0) or 0), + phase=str(d.get("phase", "")), + message=str(d.get("message", "")), + ) + + +# Cap on retained FINISHED jobs. A long-lived MCP server runs many builds over +# its lifetime; without eviction every job — and the orchestrator/client/report +# its stop-hook closure pins — would live forever (the donor's CLU-004 leak). +# Running jobs are never evicted. +_DEFAULT_MAX_FINISHED = 50 + + +class JobRegistry: + """Thread-safe registry of background jobs, keyed by ``run_id``.""" + + def __init__( + self, + max_finished: int = _DEFAULT_MAX_FINISHED, + store_path: Optional[str] = None, + ) -> None: + self._jobs: Dict[str, Job] = {} + self._lock = threading.Lock() + self._max_finished = max(1, max_finished) + # Optional durable store so run_ids/results survive an MCP server restart. + self._store = Path(store_path).expanduser() if store_path else None + if self._store is not None: + self._load() + + def _load(self) -> None: + """Load persisted jobs on startup. Best-effort: a missing/corrupt store + yields an empty registry, never an error at import/boot.""" + try: + if not self._store.exists(): + return + import json + + data = json.loads(self._store.read_text(encoding="utf-8")) + except (OSError, ValueError) as e: + logger.warning(f"Job store unreadable ({e}); starting empty.") + return + for d in data if isinstance(data, list) else []: + job = Job.from_dict(d) + if job.run_id: + self._jobs[job.run_id] = job + + def _save_locked(self) -> None: + """Persist all jobs to the store (atomic). Caller holds ``self._lock``. + Best-effort: a write failure is logged, never raised into a job's path.""" + if self._store is None: + return + try: + import json + import os + + self._store.parent.mkdir(parents=True, exist_ok=True) + payload = json.dumps([j.to_dict() for j in self._jobs.values()], indent=2) + tmp = self._store.with_suffix(self._store.suffix + ".tmp") + tmp.write_text(payload, encoding="utf-8") + os.replace(tmp, self._store) + except OSError as e: + logger.warning(f"Could not persist job store (non-fatal): {e}") + + def _prune_finished_locked(self) -> None: + """Evict the least-recently-FINISHED jobs beyond the retention cap. + + Caller must hold ``self._lock``. Ordered by ``ended_at`` (finish time), + not insertion order: a long job that started first but finished last is + the newest result and must be kept, not dropped. + """ + finished = sorted( + ( + (j.ended_at or "", rid) + for rid, j in self._jobs.items() + if j.status != "running" + ) + ) + excess = len(finished) - self._max_finished + for _, rid in finished[:excess] if excess > 0 else []: + del self._jobs[rid] + + def start( + self, + kind: str, + project_path: str, + target: Callable[[], str], + stop_hook: Optional[Callable[[], None]] = None, + ) -> str: + """Launch ``target`` in a daemon thread and return its ``run_id``. + + ``target`` runs the (synchronous) build/run and returns a report string. + Raises ``RuntimeError`` if a job is already running for ``project_path`` + — one writer per project, so concurrent runs cannot corrupt its tree. + """ + with self._lock: + for j in self._jobs.values(): + if j.status == "running" and _same_path(j.project_path, project_path): + raise RuntimeError( + f"a {j.kind} job (run_id {j.run_id}) is already running for " + f"{project_path}; stop it before starting another" + ) + run_id = uuid.uuid4().hex[:12] + job = Job( + run_id=run_id, + kind=kind, + project_path=str(project_path), + _stop_hook=stop_hook, + ) + self._jobs[run_id] = job + # Bound retention so finished jobs (and the resources their closures + # pin) don't accumulate over the server's lifetime. + self._prune_finished_locked() + self._save_locked() + + # A target may optionally accept a progress reporter: a 1-arg target gets + # ``report(**fields)`` bound to this run; a legacy 0-arg target is called + # as before (fully backward compatible). + try: + import inspect + + wants_reporter = len(inspect.signature(target).parameters) >= 1 + except (TypeError, ValueError): + wants_reporter = False + + def _report(**fields: Any) -> None: + self.update_progress(run_id, **fields) + + def _runner() -> None: + try: + report = target(_report) if wants_reporter else target() + with self._lock: + if job.stop_requested: + # build() catches the stop's budget-trip and returns its + # report normally; label it so the reader knows the run + # was cancelled, not that a real budget ran out. + job.status = "stopped" + job.result = "Stopped by request (partial).\n\n" + report + else: + job.status = "succeeded" + job.result = report + except Exception as e: # noqa: BLE001 - a crashed run must be observable, not lost + logger.error(f"Job {run_id} ({kind}) raised: {e}") + with self._lock: + if job.stop_requested: + job.status = "stopped" + job.result = f"Stopped by request; partial: {e}" + else: + job.status = "failed" + job.error = str(e) + finally: + with self._lock: + job.ended_at = _now() + self._prune_finished_locked() + self._save_locked() + + thread = threading.Thread( + target=_runner, name=f"misterdev-job-{run_id}", daemon=True + ) + job._thread = thread + thread.start() + return run_id + + def get(self, run_id: str) -> Optional[Job]: + with self._lock: + return self._jobs.get(run_id) + + def status(self, run_id: str) -> Optional[Dict[str, Any]]: + job = self.get(run_id) + return job.to_dict() if job else None + + def update_progress( + self, + run_id: str, + *, + done: Optional[int] = None, + total: Optional[int] = None, + phase: Optional[str] = None, + message: Optional[str] = None, + ) -> bool: + """Record task-level progress for a running job (surfaced by ``status``). + + Returns False for an unknown id. Only the provided fields change, so a + caller can update just the phase, or just the done count. Persisted so + progress survives a restart too.""" + with self._lock: + job = self._jobs.get(run_id) + if job is None: + return False + if done is not None: + job.tasks_done = int(done) + if total is not None: + job.tasks_total = int(total) + if phase is not None: + job.phase = str(phase) + if message is not None: + job.message = str(message) + self._save_locked() + return True + + def stop(self, run_id: str) -> bool: + """Request cooperative cancellation of a running job. + + Returns True if a running job was signalled, False if the id is unknown + or the job already finished. Idempotent: stopping twice is harmless. + """ + with self._lock: + job = self._jobs.get(run_id) + if job is None or job.status != "running": + return False + job.stop_requested = True + hook = job._stop_hook + self._save_locked() + if hook is not None: + try: + hook() + except Exception as e: # noqa: BLE001 - stop must never raise into the caller + logger.warning(f"Stop hook for job {run_id} failed (non-fatal): {e}") + return True + + def list_jobs(self) -> List[Dict[str, Any]]: + with self._lock: + return [j.to_dict() for j in self._jobs.values()] + + +def _default_job_store() -> str: + """The durable job store path: ``$MISTERDEV_STATE_DIR/jobs.json`` or the + conventional ``~/.misterdev/jobs.json`` (matching the project registry).""" + import os + + base = os.environ.get("MISTERDEV_STATE_DIR") or str(Path.home() / ".misterdev") + return str(Path(base) / "jobs.json") + + +# Process-wide registry shared by the MCP server's async tools. Persistent by +# default so run_ids/results survive a server restart; loading is best-effort and +# writes are deferred until a job actually runs. +registry = JobRegistry(store_path=_default_job_store()) diff --git a/misterdev/core/execution/parallel.py b/misterdev/core/execution/parallel.py new file mode 100644 index 0000000..7a9f6e2 --- /dev/null +++ b/misterdev/core/execution/parallel.py @@ -0,0 +1,466 @@ +"""Parallel / worktree task execution, extracted from the orchestrator. + +``ProjectOrchestrator`` mixes this in: every method uses only ``self`` (all of its +collaborators live here or on the coordinator) plus module-level helpers, so the +split is behavior-preserving — method resolution and call sites are unchanged. +The unit owns how a wave of ready tasks runs concurrently: worktree isolation +(prime by CoW clone or install, sanity-probe, conflict-free sub-wave batching), +serial merge-back with a post-merge base gate, and the shared-tree fallback. +""" + +import concurrent.futures +from typing import Optional + +from misterdev.agent_helpers import ( + _WorktreeProjectView, + worktree_healthcheck_command, + worktree_setup_command, +) +from misterdev.config import get_setting +from misterdev.core.execution.project import Project +from misterdev.core.execution.wave_partition import partition_parallel_safe +from misterdev.core.models import Task +from misterdev.logging_setup import setup_logger +from misterdev.task_executors.markdown_plan_executor import MarkdownPlanExecutor + +logger = setup_logger(__name__) + + +class ParallelExecutionMixin: + @staticmethod + def _task_file_set(task: Task) -> set: + """Declared files a task will touch (modify + create). + + Tolerates non-list values (e.g. unconfigured mocks): only real lists + contribute paths, anything else is treated as "unknown / no claim". + """ + files: set = set() + for attr in ("files_to_modify", "files_to_create"): + value = getattr(task, attr, None) + if isinstance(value, list): + files.update(str(p) for p in value) + return files + + @classmethod + def _partition_disjoint(cls, ready: list[Task]) -> tuple[list, list]: + """Split tasks into a concurrent-safe group + a serial remainder. + + A task joins the concurrent group only if its declared file set is + disjoint from every task already in that group; otherwise it is + deferred to the serial remainder so overlapping writes can't interleave. + """ + concurrent_group: list = [] + serial_remainder: list = [] + claimed: set = set() + for task in ready: + files = cls._task_file_set(task) + if files & claimed: + serial_remainder.append(task) + else: + concurrent_group.append(task) + claimed |= files + return concurrent_group, serial_remainder + + def _execute_parallel( + self, ready: list[Task], executor: MarkdownPlanExecutor, project: Project + ) -> list: + """Execute a batch of independent tasks concurrently. + + In "worktree" mode each task runs in its own git worktree so parallel + edits can't collide. When the mode is left at its default and the + project is a git repo, worktree isolation is preferred automatically; + "shared" must be requested explicitly to opt out. In shared mode only + tasks with disjoint declared file sets run in the same concurrent batch; + tasks whose file sets overlap are run serially afterwards. + """ + mode = get_setting(project.config, "orchestrator", "parallel_mode") + is_git_repo = (project.path / ".git").exists() is True + # "auto" (default) isolates via worktrees on a git repo; the value itself + # carries the intent, so no fragile "was it explicitly set" detection. + prefer_worktrees = mode == "worktree" or (mode == "auto" and is_git_repo) + if prefer_worktrees and is_git_repo: + return self._execute_parallel_worktrees(ready, executor, project) + + concurrent_group, serial_remainder = self._partition_disjoint(ready) + results = [] + max_workers = get_setting(project.config, "orchestrator", "max_workers") + if concurrent_group: + with concurrent.futures.ThreadPoolExecutor( + max_workers=min(len(concurrent_group), max_workers) + ) as pool: + future_to_task = { + pool.submit( + executor.execute, task, project, use_git_branch=False + ): task + for task in concurrent_group + } + for future in concurrent.futures.as_completed(future_to_task): + task = future_to_task[future] + try: + result = future.result() + results.append((task, result, None)) + except Exception as e: + results.append((task, None, e)) + + # Tasks with overlapping file claims run one at a time. + for task in serial_remainder: + try: + result = executor.execute(task, project, use_git_branch=False) + results.append((task, result, None)) + except Exception as e: + results.append((task, None, e)) + return results + + def _worktree_setup_command(self, project: Project) -> Optional[str]: + """The command that primes a fresh worktree's dependencies before gating, + or None to skip. Delegates to the shared resolver so worktree creation and + the per-gate infra-reprime helper agree on one command.""" + return worktree_setup_command(project.config, project.path) + + def _worktree_healthcheck_command(self, project: Project) -> Optional[str]: + """The fast probe that confirms a primed worktree's toolchain resolves, or + None to skip. Delegates to the shared resolver (auto-detected for node/pnpm + projects; overridable via ``orchestrator.worktree_healthcheck_command``).""" + return worktree_healthcheck_command(project.config, project.path) + + def _prepare_task_worktree( + self, + project, + git, + task, + wt_root, + clone_deps, + health_cmd, + setup_cmd, + setup_timeout, + cmd_tool, + ): + """Create + prime one task's worktree. Returns ``(prep, error)`` where + ``prep`` is ``(task, wt_path, branch)`` on success and ``error`` is set on + failure (with the worktree already torn down). + + A prep step (clone-prime / install / healthcheck) that RAISES after the + worktree was created would otherwise leak that worktree — the wave cleanup + only iterates fully-prepared tasks. Tear it down here so the leak can't + happen, and surface the task as errored instead of aborting the batch. + """ + import uuid + + # A run-unique branch name (never a bare ``task/``): a leftover branch + # from a prior failed run must not collide with this run's ``-b`` create. + # The unique branch is cut fresh from HEAD, so it carries the latest + # committed work (including earlier sub-waves). + run_id = uuid.uuid4().hex[:6] + branch = f"task/{task.id}-{run_id}" + wt_path = wt_root / f"{task.id}-{run_id}" + ok, out = git.worktree_add(project, str(wt_path), branch, new_branch=True) + if not ok: + logger.error(f"Worktree add failed for {task.id}: {out}") + return None, RuntimeError(f"worktree add failed: {out}") + try: + # Prime deps ONCE here (serially, off the parallel gate path) so the + # gate tests code, not install speed. Prefer a near-instant CoW clone + # of the base node_modules; that path self-verifies with the probe. + primed = False + if clone_deps: + primed = self._prime_worktree_by_clone( + project, task, wt_path, health_cmd, setup_timeout, cmd_tool + ) + # Install fallback: clone unavailable/declined or it failed the probe. + # Best-effort — the gate's own implicit install is the backstop. + if not primed and cmd_tool is not None and setup_cmd: + sok, sout = cmd_tool.execute( + project, setup_cmd, cwd=str(wt_path), timeout=setup_timeout + ) + if not sok: + logger.warning( + f"Worktree dep prep failed for {task.id} " + f"(gate will fall back to its own install): {sout[-200:]}" + ) + # A clone already passed the sanity probe; only the install path needs + # the re-prime-once healthcheck. + if not primed: + self._worktree_healthcheck( + project, + task, + wt_path, + setup_cmd, + health_cmd, + setup_timeout, + cmd_tool, + ) + return (task, wt_path, branch), None + except Exception as e: + logger.error( + f"Worktree prep raised for {task.id}; tearing down to avoid a leak: {e}" + ) + git.worktree_remove(project, str(wt_path)) + git.branch_delete(project, branch) + return None, e + + def _worktree_healthcheck( + self, project, task, wt_path, setup_cmd, health_cmd, timeout, cmd_tool + ) -> None: + """Confirm the primed worktree's toolchain resolves; heal once or flag it. + + Runs the cheap probe right after priming. On failure, a broken/partial + install is the likely cause, so re-prime the deps ONCE and re-probe. If it + still fails, log clearly that THIS WORKTREE's environment is unhealthy — + not the task's code — so a downstream gate failure here is read as an + environment fault, not attributed to the task. Best-effort: never raises + and never drops the task; the gate's own infra self-heal is the backstop. + """ + if not health_cmd or cmd_tool is None: + return + ok, out = cmd_tool.execute( + project, health_cmd, cwd=str(wt_path), timeout=timeout + ) + if ok: + return + logger.warning( + f"Worktree health probe failed for {task.id} ({health_cmd!r}); " + f"re-priming deps once and re-probing: {out[-200:]}" + ) + if setup_cmd: + cmd_tool.execute(project, setup_cmd, cwd=str(wt_path), timeout=timeout) + ok, out = cmd_tool.execute( + project, health_cmd, cwd=str(wt_path), timeout=timeout + ) + if not ok: + logger.error( + f"Worktree ENVIRONMENT unhealthy for {task.id} after re-prime " + f"({health_cmd!r}): the toolchain does not resolve in this " + f"worktree. Downstream gate failures here are an environment " + f"fault, NOT the task's code: {out[-200:]}" + ) + + def _prime_worktree_by_clone( + self, project, task, wt_path, health_cmd, timeout, cmd_tool + ) -> bool: + """Prime a worktree by copy-on-write cloning the base node_modules. + + Returns True only when the clone succeeded AND the cloned toolchain passes + the P3 sanity probe (so a cloned-but-broken tree falls back to install). + Requires a probe command (``health_cmd``) to verify with; without one we + cannot confirm the clone, so we decline and let the install path run. + """ + from misterdev.core.execution.dep_clone import clone_dependencies + + if not health_cmd or cmd_tool is None: + return False + cloned, dirs = clone_dependencies(project.path, wt_path) + if not cloned: + return False + ok, out = cmd_tool.execute( + project, health_cmd, cwd=str(wt_path), timeout=timeout + ) + if not ok: + logger.info( + f"Cloned deps for {task.id} failed the sanity probe " + f"({health_cmd!r}); falling back to install: {out[-200:]}" + ) + return False + logger.info( + f"Primed {task.id} by cloning {len(dirs)} node_modules dir(s) from the " + "base checkout (copy-on-write, no install)." + ) + return True + + def _post_merge_healthcheck( + self, + project: Project, + executor: MarkdownPlanExecutor, + git, + task: Task, + timeout: int, + ) -> bool: + """Gate the base branch after a task's merge; roll the merge back if broken. + + Runs the merged task's OWNING-target gate (typecheck/test, resolved via the + same routing the executor uses) on the base checkout. Returns True when the + base is healthy (or there is nothing to gate). On a real (non-infra) + failure the merge broke the base, so ``reset --hard HEAD^`` removes the + merge commit (it is the ``--no-ff`` tip) and returns False — the caller + then treats the task as unfinished. A transient/infra failure is NOT rolled + back: it is an environment fault, not a code break, so the merge stands. + """ + from misterdev.core.planning.targets import select_target, target_commands + from misterdev.core.execution.infra import infra_failure + + files = list(task.files_to_modify) + list(task.files_to_create) + targets = project.config.get("targets") or [] + tgt = select_target(targets, files) + cmds = target_commands(tgt, project.config) + # Prefer the cheapest reliable signal (typecheck) so a per-merge gate is + # fast; fall back to test then build when the target declares no typecheck. + gate_cmd = ( + cmds["typecheck_command"] or cmds["test_command"] or cmds["build_command"] + ) + if not gate_cmd: + return True + tp = (tgt.get("path") or "").strip("/") if tgt else "" + run_dir = project.path / tp if tp else project.path + ok, out = executor._run_command(project, gate_cmd, timeout=timeout, cwd=run_dir) + if ok: + return True + infra = infra_failure(out) + if infra: + logger.warning( + f"Post-merge gate for {task.id} failed on an environment fault " + f"({infra}), not the code; leaving the merge in place: {out[-200:]}" + ) + return True + logger.error( + f"Post-merge gate for {task.id} FAILED on the base branch " + f"({gate_cmd!r}): the merge broke the base. Rolling it back and " + f"re-queuing the task: {out[-200:]}" + ) + rok, rout = git.reset_hard(project, "HEAD^") + if not rok: + logger.error( + f"Failed to roll back {task.id}'s merge; base may be left broken: " + f"{rout[-200:]}" + ) + return False + + def _execute_parallel_worktrees( + self, ready: list[Task], executor: MarkdownPlanExecutor, project: Project + ) -> list: + """Run each task in an isolated git worktree, then merge successes back. + + Worktrees are created and merged serially (git's index/worktree metadata + is not concurrency-safe); only the task bodies run in parallel. Each fresh + worktree's dependencies are primed once at creation (see + ``_worktree_setup_command``) so the parallel gates test code, not install + speed. + """ + from misterdev.tools.command import CommandTool + from misterdev.tools.git_tool import GitTool + + git = GitTool({}) + wt_root = project.path / ".orchestrator" / "worktrees" + wt_root.mkdir(parents=True, exist_ok=True) + # Drop metadata from any worktree a prior run left dangling before we add new ones. + git.worktree_prune(project) + setup_cmd = self._worktree_setup_command(project) + setup_timeout = get_setting( + project.config, "orchestrator", "worktree_setup_timeout" + ) + health_cmd = self._worktree_healthcheck_command(project) + cmd_tool = CommandTool({}) if (setup_cmd or health_cmd) else None + post_merge_hc = get_setting( + project.config, "orchestrator", "post_merge_healthcheck" + ) + gate_timeout = get_setting(project.config, "build", "test_timeout") + # Prefer a copy-on-write clone of the base node_modules over reinstalling. + # Resolve FS support ONCE (it is the same for every worktree); a non-CoW + # filesystem (e.g. HFS+) or a missing probe transparently falls back below. + from misterdev.core.execution.dep_clone import clone_supported + + clone_deps = ( + get_setting(project.config, "orchestrator", "worktree_clone_deps") + and bool(setup_cmd) + and bool(health_cmd) + and clone_supported(project.path) + ) + results: list = [] + max_workers = get_setting(project.config, "orchestrator", "max_workers") + + def run_one(item): + task, wt_path, branch = item + view = _WorktreeProjectView(project, wt_path) + try: + return ( + task, + executor.execute(task, view, use_git_branch=False), + None, + wt_path, + branch, + ) + except Exception as e: + return (task, None, e, wt_path, branch) + + # Split the wave so tasks that DECLARE a shared file run in different + # sub-waves (serially): parallel worktrees editing the same file would + # race and clobber or conflict on merge. Disjoint tasks stay in one batch. + # Sub-waves run in order, so a later batch's worktrees are cut from HEAD + # AFTER the earlier batch merged — it builds on that work, not around it. + if get_setting(project.config, "orchestrator", "serialize_conflicting_tasks"): + batches = partition_parallel_safe( + [(t, self._task_file_set(t)) for t in ready] + ) + else: + batches = [list(ready)] + if len(batches) > 1: + logger.info( + f"Wave split into {len(batches)} conflict-free sub-wave(s) by " + f"declared file overlap ({[len(b) for b in batches]} task(s) each)." + ) + + for batch in batches: + prepared: list = [] + for task in batch: + prep, err = self._prepare_task_worktree( + project, + git, + task, + wt_root, + clone_deps, + health_cmd, + setup_cmd, + setup_timeout, + cmd_tool, + ) + if prep is not None: + prepared.append(prep) + elif err is not None: + results.append((task, None, err)) + + raw = [] + if prepared: + with concurrent.futures.ThreadPoolExecutor( + max_workers=min(len(prepared), max_workers) + ) as pool: + futures = [pool.submit(run_one, item) for item in prepared] + raw = [f.result() for f in concurrent.futures.as_completed(futures)] + + for task, result, error, wt_path, branch in raw: + # Remove the worktree BEFORE merging/deleting the branch: git refuses + # to delete a branch still checked out in a worktree, which otherwise + # leaks even merged branches. The task's commits live on the branch + # ref, so the merge below still sees them after the dir is gone. + git.worktree_remove(project, str(wt_path)) + merged = False + if ( + result is not None + and getattr(result, "status", None) == "completed" + ): + merged, mout = git.merge_worktree(project, branch) + if not merged: + # merge_worktree already aborted the conflicted merge, so the + # base is clean; re-queue the task (unfinished) rather than + # force-merging over another task's shared-file change. + logger.error( + f"Worktree merge conflicted for {task.id}; aborted and " + f"re-queuing (not force-merged): {mout[-200:]}" + ) + error, result = RuntimeError(f"merge conflict: {mout}"), None + elif post_merge_hc and not self._post_merge_healthcheck( + project, executor, git, task, gate_timeout + ): + # The merge broke the base branch and was rolled back; treat + # the task as unfinished (not completed) so it is retried and + # not recorded as done. The branch was already deleted by the + # (successful) merge, so no extra cleanup is needed. + error, result = ( + RuntimeError( + "post-merge health gate failed; base merge rolled back" + ), + None, + ) + # A successful merge already deleted the branch; drop any un-merged + # one so no throwaway branch accumulates or collides with a later run. + if not merged: + git.branch_delete(project, branch) + results.append((task, result, error)) + return results diff --git a/misterdev/core/execution/progress.py b/misterdev/core/execution/progress.py index 65bf0c8..7749fb4 100644 --- a/misterdev/core/execution/progress.py +++ b/misterdev/core/execution/progress.py @@ -73,6 +73,23 @@ def _load(self): self.completed = set() self.failed = set() self.hashes = {} + return + # One-shot reconciliation: completed is the single terminal state, so a + # task recorded as BOTH completed and failed (a pre-fix poisoned ledger, + # or a failure written after an earlier success) is healed by dropping + # it from failed — completed wins. Persist immediately so an existing + # poisoned progress.json self-heals on disk even if this run marks + # nothing else. + poisoned = self.failed & self.completed + if poisoned: + logger.info( + "Reconciling progress ledger: %d task(s) in both completed and " + "failed; keeping completed: %s", + len(poisoned), + ", ".join(sorted(poisoned)), + ) + self.failed -= self.completed + self._save() def _save(self): data = json.dumps( @@ -102,6 +119,14 @@ def needs_rerun(self, task_id: str, current_hash: str) -> bool: def mark_failed(self, task_id: str): with self._lock: + # completed is the single terminal state and always wins: a task that + # already succeeded must never be re-listed as failed (deferred + # persists through this same path), or the ledger reports two terminal + # states and a stale failed entry could shadow green work. A genuine + # post-success regression is caught by needs_rerun via the content + # hash, not by a failed entry. + if task_id in self.completed: + return self.failed.add(task_id) self._save() diff --git a/misterdev/core/execution/project.py b/misterdev/core/execution/project.py index 6a601b5..b1c801a 100644 --- a/misterdev/core/execution/project.py +++ b/misterdev/core/execution/project.py @@ -167,6 +167,16 @@ def _build_mcp(self): else ((curated,) if isinstance(curated, str) else tuple(curated)) ) servers.extend(select_curated(tiers)) + elif mcp_cfg.get("docs_tool", True) and not self._host_exec_isolated(): + # Mount the documentation tool (context7/fetch) by DEFAULT so the model + # looks up version-pinned library docs instead of hallucinating APIs. + # Docs-category core servers only (not the full curated stack); opt out + # with mcp.docs_tool: false, or take the whole stack via mcp.curated. + # Gated by isolation: these npx-provisioned servers fetch from the + # network on the host, refused under network=none/container like discovery. + from misterdev.core.integration.mcp_registry import select_curated + + servers.extend(select_curated(("core",), categories={"docs"})) # On-the-fly discovery: search the free official registry for servers # matching each capability query and append the trusted, locally- # runnable ones (npx/uvx stdio). Best-effort; never blocks the build. diff --git a/misterdev/core/execution/wave_partition.py b/misterdev/core/execution/wave_partition.py new file mode 100644 index 0000000..680964d --- /dev/null +++ b/misterdev/core/execution/wave_partition.py @@ -0,0 +1,42 @@ +"""Partition a wave of tasks into parallel-safe batches by declared file overlap. + +Independent tasks in a wave run concurrently in isolated worktrees, then merge +back one at a time. But two tasks that both edit a SHARED file (``env.ts``, a +route registry, a shared schema) can't safely run in parallel: their merges race +and the second either conflicts or silently clobbers the first. The plan already +declares each task's files (``files_to_create``/``files_to_modify``), so we can +detect that up front and put conflicting tasks in DIFFERENT sub-waves — run +serially — while keeping genuinely disjoint tasks parallel. + +This module is the pure decision: (item, file_set) pairs in -> list of +parallel-safe batches out. Side-effect-free so the conflict policy is directly +unit-testable; the orchestrator supplies the file sets and runs each batch. +""" + +from typing import Iterable, List, Set, Tuple, TypeVar + +T = TypeVar("T") + + +def partition_parallel_safe(items: Iterable[Tuple[T, Iterable[str]]]) -> List[List[T]]: + """Group ``(item, file_set)`` pairs into batches with pairwise-disjoint files. + + Two items whose declared file sets intersect land in DIFFERENT batches, so + they never run concurrently and can't clobber a shared file on merge; items + with disjoint sets share a batch and run in parallel. First-fit: each item + joins the earliest batch it does not conflict with, else starts a new one — + which keeps the batch count (and thus the serial depth) small. Order is + preserved within and across batches. An item with an empty/unknown file set + makes no claim and joins the first batch. + """ + batches: List[Tuple[List[T], Set[str]]] = [] + for item, files in items: + fs = {str(f) for f in (files or ())} + for batch_items, claimed in batches: + if fs.isdisjoint(claimed): + batch_items.append(item) + claimed |= fs + break + else: + batches.append(([item], set(fs))) + return [batch_items for batch_items, _ in batches] diff --git a/misterdev/core/integration/mcp_registry.py b/misterdev/core/integration/mcp_registry.py index e0d4536..2dbdcb9 100644 --- a/misterdev/core/integration/mcp_registry.py +++ b/misterdev/core/integration/mcp_registry.py @@ -80,6 +80,10 @@ # npm -> npx, PyPI -> uvx. Anything else is not auto-runnable here. _RUNTIME_BY_REGISTRY = {"npm": "npx", "pypi": "uvx"} +# The only commands a discovered server may spawn. A registry-supplied runtimeHint +# is honored ONLY if it is one of these — otherwise an untrusted entry could set +# `command` to an arbitrary binary and bypass the npx/uvx launcher sandbox. +_ALLOWED_RUNTIMES = frozenset(_RUNTIME_BY_REGISTRY.values()) _CURATED_PATH = Path(__file__).with_name("curated_servers.json") @@ -347,6 +351,14 @@ def _pinned_identifier(pkg: Dict[str, Any], runtime_hint: str) -> str: def _to_config(server: Dict[str, Any], pkg: Dict[str, Any]) -> Dict[str, Any]: runtime = _RUNTIME_BY_REGISTRY[(pkg.get("registryType") or "").lower()] runtime_hint = pkg.get("runtimeHint") or runtime + if runtime_hint not in _ALLOWED_RUNTIMES: + logger.warning( + "MCP discovery: ignoring untrusted runtimeHint %r for '%s'; using %s.", + runtime_hint, + server.get("name"), + runtime, + ) + runtime_hint = runtime args = _positional_values(pkg.get("runtimeArguments")) if runtime_hint == "npx" and "-y" not in args: args = ["-y", *args] @@ -517,6 +529,7 @@ def select_curated( tiers=("core",), env: Optional[Dict[str, str]] = None, catalog: Optional[List[Dict[str, Any]]] = None, + categories: Optional[set] = None, ) -> List[Dict[str, Any]]: """Curated stdio configs for the requested tiers. @@ -524,6 +537,8 @@ def select_curated( manual Go/Docker/OAuth entry) AND whose ``requires`` env vars are all present, so a keyed server (e.g. Postgres needing ``DATABASE_URI``) mounts only when it can actually run. ``tiers`` accepts ``"all"`` for every tier. + ``categories``, when given, further restricts to those catalog categories + (e.g. ``{"docs"}`` to mount only the documentation servers). """ if env is None: import os @@ -536,6 +551,8 @@ def select_curated( for entry in catalog: if want is not None and entry.get("tier") not in want: continue + if categories is not None and entry.get("category") not in categories: + continue command = entry.get("command") if not command or not isinstance(entry.get("args"), list): continue # manual/remote entry — recorded, not auto-mounted diff --git a/misterdev/core/planning/decomposer.py b/misterdev/core/planning/decomposer.py index 74252a8..bf3186d 100644 --- a/misterdev/core/planning/decomposer.py +++ b/misterdev/core/planning/decomposer.py @@ -231,13 +231,16 @@ def decompose_spec( # repo). Run before dependency detection so reclassified files participate. _ground_task_paths(tasks, project_ref) - # Detect implicit dependencies from file overlaps + # Cap BEFORE wiring implicit deps so overlap-derived edges are only drawn among + # kept tasks, and prune any explicit dep pointing at a trimmed task. + tasks = _cap_tasks(tasks, max_tasks) + + # Detect implicit dependencies from file overlaps (kept tasks only) _add_implicit_dependencies(tasks) - # Validate and cap - if len(tasks) > max_tasks: - logger.warning(f"LLM returned {len(tasks)} tasks, capping at {max_tasks}") - tasks = tasks[:max_tasks] + # Flag any task that breaks the size/verifiability invariant so the executor + # can split or reject it rather than silently trying to land it in one attempt. + enforce_task_invariants(tasks) return tasks @@ -321,6 +324,156 @@ def _ground_task_paths(tasks: list[Task], project_ref: str) -> None: t.files_to_create = still_create +def _cap_tasks(tasks: list[Task], max_tasks: int) -> list[Task]: + """Cap the task list to ``max_tasks`` and drop any dependency that now points + at a trimmed task. + + Trimming after dependency wiring would leave a surviving task depending on a + task that no longer exists; topological_sort ignores unknown deps, so the + survivor would be treated as if that (never-run) prerequisite were satisfied. + Pruning the dangling edge makes the loss explicit instead of silent. + """ + if len(tasks) <= max_tasks: + return tasks + kept = tasks[:max_tasks] + dropped = {t.id for t in tasks[max_tasks:]} + depended_on = dropped & {d for t in kept for d in t.dependencies} + logger.warning( + "Decomposer: %d tasks > cap %d; dropping %s%s.", + len(tasks), + max_tasks, + sorted(dropped), + f" (some were depended-on, edges pruned: {sorted(depended_on)})" + if depended_on + else "", + ) + kept_ids = {t.id for t in kept} + for t in kept: + t.dependencies = [d for d in t.dependencies if d in kept_ids] + return kept + + +def enforce_task_invariants(tasks: list[Task], max_files: int = 20) -> list[Task]: + """Flag decomposed tasks that violate the size/verifiability invariant. + + Pure (no LLM/no I/O). A task is well-formed when it (a) touches at most + ``max_files`` distinct files (create+modify) and (b) carries a concrete, + non-empty acceptance criterion. Violations are recorded on + ``task.processor_data["invariant_violations"]`` and logged, so downstream + (the decompose rung / keystone split) can act on an over-large or unverifiable + task instead of silently trying to land it in one attempt. + """ + # Low-signal acceptance strings that encode nothing a gate can check. + _PLACEHOLDERS = {"works", "done", "ok", "todo", "tbd", "n/a", "it works"} + for task in tasks: + violations: list[str] = [] + touched = { + f for f in (list(task.files_to_create) + list(task.files_to_modify)) if f + } + if len(touched) > max_files: + violations.append( + f"touches {len(touched)} files > max {max_files}; split it" + ) + crit = (getattr(task, "acceptance_criteria", "") or "").strip() + if len(crit) < 8 or crit.lower() in _PLACEHOLDERS: + violations.append("acceptance criteria missing or not concrete/verifiable") + if violations: + if not isinstance(task.processor_data, dict): + task.processor_data = {} + task.processor_data["invariant_violations"] = violations + logger.warning( + "Task %s violates the size/verifiability invariant: %s", + task.id, + "; ".join(violations), + ) + return tasks + + +def _partition(items: list, n: int) -> list: + """Split ``items`` into ``n`` contiguous, near-equal chunks.""" + k, m = divmod(len(items), n) + out, start = [], 0 + for i in range(n): + size = k + (1 if i < m else 0) + out.append(items[start : start + size]) + start += size + return out + + +def split_keystone_tasks( + tasks: list[Task], *, fanin_threshold: int = 3, min_units: int = 2 +) -> list[Task]: + """Proactively split a keystone (high-fan-in) task into chained sub-units. + + A keystone — a task many others depend on — landing all its behavior in one + attempt has a low per-attempt success rate AND blocks its whole fan-out on that + single attempt. When such a task touches at least ``min_units`` files it is + split BEFORE execution into smaller sub-tasks partitioned by file and chained in + order; every task that depended on the keystone is rewired to depend on the + FINAL sub-unit, so the dependency semantics are preserved (dependents still wait + for all of the keystone's work). Pure: no LLM, no I/O. A task with fewer than + ``fanin_threshold`` dependents, or too few files to partition, is unchanged. + """ + by_id = {t.id: t for t in tasks} + dependents: dict[str, list[str]] = {t.id: [] for t in tasks} + for t in tasks: + for dep in t.dependencies: + if dep in by_id: + dependents[dep].append(t.id) + + plans: dict[str, list[Task]] = {} + for t in tasks: + seen: set = set() + files = [ + f + for f in (list(t.files_to_create) + list(t.files_to_modify)) + if f and not (f in seen or seen.add(f)) + ] + if len(dependents[t.id]) < fanin_threshold or len(files) < min_units: + continue + creates = set(t.files_to_create) + chunks = _partition(files, min_units) + subs: list[Task] = [] + for i, chunk in enumerate(chunks): + subs.append( + Task( + id=f"{t.id}-part{i + 1}", + title=(t.title or t.id) + f" (part {i + 1}/{len(chunks)})", + description=t.description, + acceptance_criteria=t.acceptance_criteria, + files_to_create=[f for f in chunk if f in creates], + files_to_modify=[f for f in chunk if f not in creates], + context_files=list(t.context_files), + dependencies=( + list(t.dependencies) if i == 0 else [f"{t.id}-part{i}"] + ), + complexity=t.complexity, + category=t.category, + type=t.type, + status="pending", + project_ref=t.project_ref, + processor_data=( + dict(t.processor_data) + if isinstance(t.processor_data, dict) + else {} + ), + ) + ) + plans[t.id] = subs + + if not plans: + return tasks + + # A dependency on a split keystone now points at that keystone's FINAL sub-unit. + final_of = {kid: subs[-1].id for kid, subs in plans.items()} + result: list[Task] = [] + for t in tasks: + result.extend(plans[t.id] if t.id in plans else [t]) + for t in result: + t.dependencies = [final_of.get(d, d) for d in t.dependencies] + return result + + def _add_implicit_dependencies(tasks: list[Task]) -> None: """If task B modifies a file that task A creates, B depends on A.""" creates_map: dict[str, str] = {} diff --git a/misterdev/core/planning/plan_store.py b/misterdev/core/planning/plan_store.py new file mode 100644 index 0000000..67463b2 --- /dev/null +++ b/misterdev/core/planning/plan_store.py @@ -0,0 +1,115 @@ +"""Persisted plan proposals with an explicit approval gate. + +An MCP client can ask misterdev to *propose* work (analysis + recommendations) +without the codebase entering the client's context, review the proposals, then +*approve* a subset before any code is edited. That review step needs somewhere +to hold the proposals between calls; this module is it — a small JSON store +under the project's ``.orchestrator`` directory. + +Pure and side-effect-scoped: it only reads/writes ``proposed_plan.json`` and +carries no LLM or execution logic, so the approval semantics are fully +unit-testable offline. Analysis (which produces proposals) and execution (which +acts on approvals) live in the orchestrator. +""" + +from __future__ import annotations + +import json +from pathlib import Path +from typing import Any, Dict, List, Optional + +from misterdev.logging_setup import setup_logger + +logger = setup_logger(__name__) + +_PLAN_FILE = "proposed_plan.json" + + +def _plan_path(project_path: str | Path) -> Path: + return Path(project_path) / ".orchestrator" / _PLAN_FILE + + +def save_plan( + project_path: str | Path, recommendations: List[Any] +) -> List[Dict[str, Any]]: + """Persist ``recommendations`` as a fresh, unapproved proposed plan. + + Each recommendation may be a dict or an object with ``title`` / + ``work_type`` / ``rationale`` attributes (e.g. advisor ``Recommendation``). + Assigns stable ``P-001``-style ids and ``approved: false``, overwriting any + prior plan. Returns the stored items. + """ + items: List[Dict[str, Any]] = [] + for i, rec in enumerate(recommendations, 1): + title = _field(rec, "title") + if not title: + continue + items.append( + { + "id": f"P-{i:03d}", + "title": title, + "work_type": _field(rec, "work_type") or "complete", + "rationale": _field(rec, "rationale") or "", + "approved": False, + } + ) + _write(project_path, items) + return items + + +def load_plan(project_path: str | Path) -> Optional[List[Dict[str, Any]]]: + """Return the stored plan items, or None when no plan has been proposed.""" + path = _plan_path(project_path) + if not path.is_file(): + return None + try: + data = json.loads(path.read_text(encoding="utf-8")) + except (OSError, ValueError) as e: + logger.error(f"Could not read proposed plan at {path}: {e}") + return None + return data if isinstance(data, list) else None + + +def set_approval( + project_path: str | Path, + item_ids: Optional[List[str]] = None, + approve_all: bool = False, + reject_ids: Optional[List[str]] = None, +) -> Optional[List[Dict[str, Any]]]: + """Update approval flags on the stored plan and persist. + + ``approve_all`` approves every item. Otherwise ``item_ids`` are approved and + ``reject_ids`` are un-approved; an id in both is rejected (reject wins, the + safer default). Returns the updated items, or None if no plan exists. + """ + plan = load_plan(project_path) + if plan is None: + return None + approve = set(item_ids or []) + reject = set(reject_ids or []) + for item in plan: + if approve_all: + item["approved"] = True + if item["id"] in approve: + item["approved"] = True + if item["id"] in reject: + item["approved"] = False + _write(project_path, plan) + return plan + + +def approved_items(project_path: str | Path) -> List[Dict[str, Any]]: + """Return only the approved items (empty list when none / no plan).""" + plan = load_plan(project_path) or [] + return [item for item in plan if item.get("approved")] + + +def _field(rec: Any, name: str) -> str: + value = rec.get(name) if isinstance(rec, dict) else getattr(rec, name, None) + return str(value).strip() if value else "" + + +def _write(project_path: str | Path, items: List[Dict[str, Any]]) -> None: + path = _plan_path(project_path) + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(json.dumps(items, indent=2), encoding="utf-8") diff --git a/misterdev/core/planning/requirements.py b/misterdev/core/planning/requirements.py index e551346..8132f84 100644 --- a/misterdev/core/planning/requirements.py +++ b/misterdev/core/planning/requirements.py @@ -140,7 +140,7 @@ def gating_requirements(reqs: List[Dict], tasks, threshold: int = 3) -> List[Dic never gate on their own.""" out = [] for r in reqs: - if r.get("satisfied"): + if r.get("satisfied") or r.get("answered"): continue if r.get("kind") != "account": continue @@ -266,7 +266,7 @@ def write(self, reqs: List[Dict]) -> None: lines = [head, f"- Status: {mark}", f"- Needed by: {tids or '(plan)'}"] if r.get("how_to_provide"): lines.append(f"- Provide: {r['how_to_provide']}") - if r.get("kind") == "decision" and not r.get("satisfied"): + if not r.get("satisfied"): lines.append(f"- Answer: {existing.get(r['key'], _PLACEHOLDER)}") blocks.append("\n".join(lines)) self.md_path.write_text("\n\n".join(blocks) + "\n", encoding="utf-8") diff --git a/misterdev/core/planning/sovereign.py b/misterdev/core/planning/sovereign.py index 7b2e311..e0cf3d2 100644 --- a/misterdev/core/planning/sovereign.py +++ b/misterdev/core/planning/sovereign.py @@ -260,13 +260,21 @@ def __init__(self, project_path: Path): self._load() def _load(self): - if self.cert_file.exists(): - try: - self.data = json.loads(self.cert_file.read_text(encoding="utf-8")) - except (json.JSONDecodeError, OSError): - self.data = {"invariants": [], "decisions": []} - else: - self.data = {"invariants": [], "decisions": []} + self.data = {"invariants": [], "decisions": []} + if not self.cert_file.exists(): + return + try: + loaded = json.loads(self.cert_file.read_text(encoding="utf-8")) + except (json.JSONDecodeError, OSError): + return + # Valid JSON of the WRONG shape (a list, a scalar, or a dict missing the + # expected keys) would crash certify_decision/get_consensus_context later. + # Normalize: keep only list-valued invariants/decisions, default the rest. + if isinstance(loaded, dict): + for key in ("invariants", "decisions"): + value = loaded.get(key) + if isinstance(value, list): + self.data[key] = value def certify_decision(self, decision: str, rationale: str): """Records a certified project decision to keep future tasks aligned.""" diff --git a/misterdev/core/verification/changed_region_mutation.py b/misterdev/core/verification/changed_region_mutation.py index 5dc66a2..f4f0702 100644 --- a/misterdev/core/verification/changed_region_mutation.py +++ b/misterdev/core/verification/changed_region_mutation.py @@ -75,8 +75,10 @@ def changed_line_indices(old_content: str, new_content: str) -> Set[int]: ).get_opcodes(): if tag in ("replace", "insert"): changed.update(range(j1, j2)) - if not changed and new: - # Whole-file replacement with no diffable anchor: treat all as changed. + if not changed and new and not old: + # A brand-NEW file (no prior content): every line is new. An UNCHANGED file + # (old == new) legitimately has no changed region — do not treat it as + # fully changed, which would mutate untouched code. changed = set(range(len(new))) return changed diff --git a/misterdev/core/verification/gatekeeper/__init__.py b/misterdev/core/verification/gatekeeper/__init__.py index 7e85c35..7319ce3 100644 --- a/misterdev/core/verification/gatekeeper/__init__.py +++ b/misterdev/core/verification/gatekeeper/__init__.py @@ -152,6 +152,9 @@ def run_gates( self, commands: Dict[str, Optional[str]] ) -> Tuple[bool, List[str], HealthCheck]: """Run the gate sequence. Returns (success, issues, final_health).""" + # The tree may have changed since the last run_gates; recompute the diff + # once for this run (G5/G6/G9 then share the memoized result). + self._diff_cache_valid = False issues: List[str] = [] health = HealthCheck() @@ -516,6 +519,19 @@ def _scan_secrets_whole_tree(self) -> List[str]: return flagged def _iter_diff_added_lines(self) -> Optional[List[Tuple[str, str]]]: + """Memoized ``(file_path, added_line)`` diff pairs for THIS run_gates. + + G5/G6/G9 all consume the same union diff, and the tree does not change + between gates within one run_gates call, so the 3+ git subprocesses and + untracked-file reads are computed once and reused. ``run_gates`` clears the + cache at its start so a later run over a changed tree recomputes.""" + if getattr(self, "_diff_cache_valid", False): + return self._diff_cache + self._diff_cache = self._compute_diff_added_lines() + self._diff_cache_valid = True + return self._diff_cache + + def _compute_diff_added_lines(self) -> Optional[List[Tuple[str, str]]]: """Return ``(file_path, added_line)`` pairs from the git diff. Covers both staged and unstaged changes against HEAD (the union of diff --git a/misterdev/core/verification/held_out_oracle.py b/misterdev/core/verification/held_out_oracle.py deleted file mode 100644 index b2e12fe..0000000 --- a/misterdev/core/verification/held_out_oracle.py +++ /dev/null @@ -1,128 +0,0 @@ -"""Held-out oracle: is a per-task acceptance check actually NON-TRIVIAL? - -The strongest per-task oracle is a test that fails before the fix and passes -after (see spec_tests.py). But a generated or hand-written acceptance test can be -*trivial* — it passes even against a do-nothing implementation (e.g. an assert -that the function merely returns a string). A trivial oracle is how a -runtime-broken fix still "passes" and merges (docs/research-directions.md, -Theme 1 — Cogeneration 2601.19066). - -This module gives the missing guarantee: STUB the fix's changed region (blank the -edited function bodies) and re-run the acceptance test. If the test still passes -against the stub, it does not actually constrain the behavior — reject it as a -weak oracle rather than trust the pass. If the stub fails the test, the oracle is -real: passing it means something. - -Pure + injectable (a ``runner`` runs the test; a ``writer`` swaps file contents), -so it is fully testable with no subprocess. Best-effort and self-restoring: the -original file is always put back, and any failure degrades to "cannot judge" -(SKIP) rather than raising into the build. -""" - -import ast -import re -from pathlib import Path -from typing import Callable, Optional, Tuple - -from misterdev.core.execution.outcomes import GREEN, RED, SKIP -from misterdev.core.verification.changed_region_mutation import changed_line_indices -from misterdev.core.verification.mutation_gate import MutationResult -from misterdev.logging_setup import setup_logger - -logger = setup_logger(__name__) - -# Runner: (test_command, timeout) -> (passed, output). Same shape as the -# changed-region mutation runner, so callers can share one. -Runner = Callable[[str, float], Tuple[bool, str]] - - -def stub_python(source: str, changed_lines) -> Optional[str]: - """Blank the body of every function/method that overlaps a changed line, - replacing it with ``raise NotImplementedError``. Returns None when nothing - could be stubbed (no changed function) or the source does not parse.""" - try: - tree = ast.parse(source) - except SyntaxError: - return None - lines = source.splitlines(keepends=True) - changed = set(changed_lines) - # Collect (start, end, indent) for each edited function, outermost-first so a - # later replacement never lands inside an already-replaced span. - spans = [] - for node in ast.walk(tree): - if not isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)): - continue - start = node.body[0].lineno - 1 # 0-based first body line - end = (node.end_lineno or node.body[-1].lineno) - 1 - if any(start <= ln <= end for ln in changed): - indent = " " * (node.body[0].col_offset) - spans.append((start, end, indent)) - if not spans: - return None - spans.sort(key=lambda s: s[0], reverse=True) # replace bottom-up - for start, end, indent in spans: - lines[start : end + 1] = [f"{indent}raise NotImplementedError\n"] - return "".join(lines) - - -def _stub_for(path: str, old: str, new: str) -> Optional[str]: - changed = changed_line_indices(old, new) - if path.endswith(".py"): - return stub_python(new, changed) - return None # only Python stubbing is implemented; other languages SKIP - - -def check_oracle( - project_path: Path, - rel_path: str, - old_content: str, - new_content: str, - test_command: str, - runner: Runner, - timeout: float = 120.0, -) -> MutationResult: - """RED when the acceptance test still PASSES against a stubbed fix (trivial - oracle — the test does not constrain behavior); GREEN when the stub fails it - (real oracle). SKIP when no stub can be formed or no test command exists. - Always restores the file.""" - if not test_command: - return MutationResult(SKIP, reason="no test command") - stub = _stub_for(rel_path, old_content, new_content) - if stub is None or stub == new_content: - return MutationResult(SKIP, reason="no stubbable changed function") - target = project_path / rel_path - try: - target.write_text(stub, encoding="utf-8") - passed, _out = runner(test_command, timeout) - except OSError as e: - return MutationResult(SKIP, reason=f"oracle check error: {e}") - finally: - try: - target.write_text(new_content, encoding="utf-8") - except OSError as e: # restoration must not be silent — it changed source - logger.error(f"held-out oracle: FAILED to restore {rel_path}: {e}") - if passed: - return MutationResult( - RED, - score=0.0, - reason="acceptance test passes against a do-nothing stub — a trivial " - "oracle that does not verify the fix", - ) - return MutationResult( - GREEN, score=1.0, reason="oracle rejects the stub (non-trivial)" - ) - - -_TRIVIAL_ASSERT_RE = re.compile( - r"assert\s+(isinstance\(|.*\bis\s+not\s+None\b|len\(.*\)\s*>?=?\s*\d|True\b)", -) - - -def looks_trivial(test_source: str) -> bool: - """Cheap static smell test: a test whose only assertions are shape checks - (isinstance / not-None / len>0 / assert True) is likely a weak oracle. A - zero-cost pre-filter before the (more expensive) stub run.""" - asserts = [ln for ln in test_source.splitlines() if "assert" in ln] - if not asserts: - return True - return all(_TRIVIAL_ASSERT_RE.search(ln) for ln in asserts) diff --git a/misterdev/core/verification/independent.py b/misterdev/core/verification/independent.py index b21f87f..fe6101e 100644 --- a/misterdev/core/verification/independent.py +++ b/misterdev/core/verification/independent.py @@ -52,11 +52,22 @@ def build_independent_call( return None can_switch = hasattr(llm_client, "with_model") + # A judge model that equals the generator's own model is NOT independence: it + # routes to the same weights and shares the same blind spots. Detect it so the + # weaker case is surfaced instead of silently masquerading as an independent + # check, and do not bother switching to the identical model. + same_model = bool(model) and model == getattr(llm_client, "model", None) if model and not can_switch: logger.warning( f"{role}: an independent model is set but the client cannot switch " f"models; running on the generator's own model (weaker independence)." ) + elif same_model: + logger.warning( + f"{role}: the configured model ({model}) is the SAME as the generator's " + f"model, so this judgment shares the generator's blind spots (no real " + f"independence). Set a DIFFERENT model for a stronger, independent check." + ) elif not model and role not in _INDEPENDENCE_NOTED: _INDEPENDENCE_NOTED.add(role) logger.info( @@ -65,7 +76,7 @@ def build_independent_call( f"set a different model for it in project.yaml. Noted once per run." ) - effective = model if (model and can_switch) else None + effective = model if (model and can_switch and not same_model) else None def _call(prompt: str) -> str: return generate_independent(llm_client, prompt, system, model=effective) diff --git a/misterdev/core/verification/validator.py b/misterdev/core/verification/validator.py index c3d0311..5deffa4 100644 --- a/misterdev/core/verification/validator.py +++ b/misterdev/core/verification/validator.py @@ -305,15 +305,17 @@ def _parse_test_counts(output: str) -> Tuple[int, int]: if m: f, total = int(m.group(1)), int(m.group(2)) return total, f - # dotnet test (VSTest): "Failed: N, Passed: M, Skipped: K, Total: T" - fm = re.search(r"Failed:\s*(\d+)", output) - tm = re.search(r"Total:\s*(\d+)", output) - if fm and tm: - return int(tm.group(1)), int(fm.group(1)) - # dotnet test (alt): "Total tests: T. Passed: M. Failed: N." - m = re.search(r"Total tests:\s*(\d+)\..*?Failed:\s*(\d+)", output, re.DOTALL) - if m: - return int(m.group(1)), int(m.group(2)) + # dotnet test (VSTest): "Failed: N, Passed: M, Skipped: K, Total: T" — SUM every + # project's block, like the pytest/cargo branches, so a multi-project solution is + # not undercounted to its first project (which would hide a regression). + fails = re.findall(r"Failed:\s*(\d+)", output) + totals = re.findall(r"Total:\s*(\d+)", output) + if fails and totals: + return sum(int(n) for n in totals), sum(int(n) for n in fails) + # dotnet test (alt): "Total tests: T. Passed: M. Failed: N." — sum all blocks. + alt = re.findall(r"Total tests:\s*(\d+)\..*?Failed:\s*(\d+)", output, re.DOTALL) + if alt: + return sum(int(t) for t, _ in alt), sum(int(f) for _, f in alt) # node --test: "ℹ tests N" / "ℹ fail K" (default reporter) or the TAP # equivalent "# tests N" / "# fail K". tm = re.search(r"(?:ℹ|#)\s*tests\s+(\d+)", output) diff --git a/misterdev/core/verification/web_verify.py b/misterdev/core/verification/web_verify.py index 1e30a4e..97ee818 100644 --- a/misterdev/core/verification/web_verify.py +++ b/misterdev/core/verification/web_verify.py @@ -25,6 +25,7 @@ check is treated as seeded (SKIP-like), not RED. """ +import select import subprocess import time from pathlib import Path @@ -171,21 +172,36 @@ def _start_server( text=True, bufsize=1, ) - if ready: - captured: List[str] = [] - while time.monotonic() < deadline: - if "".join(captured).find(ready) != -1: - break - if proc.poll() is not None: - break - if proc.stdout is not None: - line = proc.stdout.readline() - if line: - captured.append(line) - else: - # No readiness signal: give the server a brief moment to bind its port. - time.sleep(min(2.0, max(0.0, deadline - time.monotonic()))) - return proc + try: + if ready: + captured: List[str] = [] + while time.monotonic() < deadline: + if "".join(captured).find(ready) != -1: + break + if proc.poll() is not None: + break + if proc.stdout is None: + break + # Bounded wait: a raw readline() blocks until the server emits a + # line, which for a silent server never returns and would run past + # the deadline (leaking this process past the gate's outer bound). + # select re-checks the deadline at most every 0.5s. + rlist, _, _ = select.select([proc.stdout], [], [], 0.5) + if rlist: + line = proc.stdout.readline() + if line: + captured.append(line) + elif proc.poll() is not None: + break + else: + # No readiness signal: give the server a brief moment to bind its port. + time.sleep(min(2.0, max(0.0, deadline - time.monotonic()))) + return proc + except Exception: + # A failure while waiting (e.g. select unsupported on this platform) must + # not orphan the process we just launched. + _terminate(proc) + raise def _drive_browser( diff --git a/misterdev/environments/venv_env.py b/misterdev/environments/venv_env.py index b23db70..a0d6aca 100644 --- a/misterdev/environments/venv_env.py +++ b/misterdev/environments/venv_env.py @@ -24,13 +24,27 @@ def setup(self) -> bool: "setup_commands", [f"python -m venv {self.root_dir}"] ) + timeout = float(self.config.get("setup_timeout", 600)) for cmd_template in setup_commands: command = cmd_template.format(root_dir=self.root_dir) logger.info(f"Running env setup command: {command}") try: - subprocess.run(command, shell=True, cwd=self.project_path, check=True) + subprocess.run( + command, + shell=True, + cwd=self.project_path, + check=True, + capture_output=True, + text=True, + timeout=timeout, + ) + except subprocess.TimeoutExpired: + logger.error(f"Env setup command timed out after {timeout}s: {command}") + return False except subprocess.CalledProcessError as e: - logger.error(f"Failed to setup environment: {e}") + logger.error( + f"Failed to setup environment: {e}; {e.stderr or e.stdout or ''}" + ) return False return True diff --git a/misterdev/mcp_server.py b/misterdev/mcp_server.py index 7fbd7e5..da3effc 100644 --- a/misterdev/mcp_server.py +++ b/misterdev/mcp_server.py @@ -13,6 +13,7 @@ idempotent, open-world) so an AI agent can pick and call them correctly. """ +from pathlib import Path from typing import Annotated, Any, Dict, Optional from mcp.server.fastmcp import FastMCP @@ -20,6 +21,9 @@ from pydantic import Field from misterdev.agent import ProjectOrchestrator +from misterdev.core.execution.jobs import registry +from misterdev.core.planning.plan_store import load_plan, set_approval +from misterdev.core.reporting.report_view import collect # Conservative default $ ceiling for an AI-client-triggered build (the CLI # default is higher). The client can raise it explicitly per call. @@ -91,6 +95,52 @@ def status( return ProjectOrchestrator().get_project_status(path) +@mcp.tool( + annotations=ToolAnnotations( + title="Show the latest build report, audit trail, and model stats", + readOnlyHint=True, + idempotentHint=True, + openWorldHint=False, + ) +) +def report( + path: Annotated[ + str, + Field( + description=( + "Absolute path to a project directory that has been built at " + "least once. Reads only misterdev's own ``.orchestrator`` " + "artifacts under it. Example: '/Users/me/code/my-api'." + ), + examples=["/Users/me/code/my-api"], + min_length=1, + ), + ], +) -> Dict[str, Any]: + """Return the latest build report, audit trail, and model performance. + + Use when: a ``build`` or ``run`` has finished (or was stopped) and you want + the outcome — which tasks completed/failed/deferred, per-file edits, failed + commands, governance escalations, unmet-goal gaps, token/cost totals, and + per-model success rates. This is misterdev's read-only equivalent of asking + "what did the last run find and do?". Do NOT use when: nothing has been + built yet — ``latest_report`` will be null. Related: ``status`` (live task + states), ``build``/``run`` (produce a report). + + Side effects: none — reads only the project's ``.orchestrator`` artifacts, + calls no LLM, and returns the same result on repeated calls (idempotent). + + Returns an object with ``latest_report`` (the most recent build's structured + summary, or null), ``audit`` (command/edit/governance counts), and + ``models`` (per-model attempts, success rate, and average cost). Returns an + ``error`` field instead when ``path`` is not an existing directory. + """ + proj = Path(path).expanduser() + if not proj.is_dir(): + return {"error": f"not a directory: {path}"} + return collect(proj) + + @mcp.tool( annotations=ToolAnnotations( title="Scan a directory and register the projects found", @@ -209,6 +259,19 @@ def build( ge=1, ), ] = None, + reference_dir: Annotated[ + Optional[str], + Field( + description=( + "Absolute path to a reference implementation to port from (often " + "in another language). Its module/symbol map is extracted " + "READ-ONLY and given to the planner so the build reproduces the " + "reference's design idiomatically. Omit when not porting. " + "Example: '/Users/me/code/donor-impl'." + ), + examples=["/Users/me/code/donor-impl"], + ), + ] = None, ) -> str: """Autonomously plan AND execute a goal in a project, from scratch. @@ -218,6 +281,8 @@ def build( regresses. Do NOT use when: a task plan already exists and you just want to execute it (use ``run``), or the working tree is dirty (commit/stash first). Related: ``run`` (execute an existing plan), ``status`` (inspect tasks). + Pass ``reference_dir`` to port from an existing implementation: its + module/symbol map is extracted read-only and guides the plan. DESTRUCTIVE side effects: edits files and makes git commits, and calls an external LLM provider (open-world, non-idempotent). It refuses to run on a @@ -234,7 +299,7 @@ def build( parts.append("--parallel") if max_tasks is not None: parts += ["--max-tasks", str(max_tasks)] - report = orch.build(path, " ".join(parts)) + report = orch.build(path, " ".join(parts), reference_dir=reference_dir) outcome = "succeeded" if orch.last_build_succeeded else "did not fully succeed" return f"Build {outcome}.\n\n{report}" @@ -304,6 +369,405 @@ def run( return f"{verb} pending tasks for {path}." +@mcp.tool( + annotations=ToolAnnotations( + title="Start a build in the background (returns a run_id)", + readOnlyHint=False, + destructiveHint=True, + idempotentHint=False, + openWorldHint=True, + ) +) +def build_async( + path: Annotated[ + str, + Field( + description=( + "Absolute path to the project directory to build. Its git " + "working tree must be clean. Example: '/Users/me/code/my-api'." + ), + examples=["/Users/me/code/my-api"], + min_length=1, + ), + ], + goal: Annotated[ + str, + Field( + description=( + "What to build, or a mode word ('debug', 'complete', 'review'). " + "Example: 'add rate limiting to the public API'." + ), + examples=["add rate limiting to the public API", "debug"], + min_length=1, + ), + ], + budget: Annotated[ + float, + Field(description="Maximum US dollars to spend; must be > 0.", gt=0), + ] = _DEFAULT_MCP_BUDGET, + parallel: Annotated[ + bool, + Field(description="Run independent tasks concurrently in worktrees."), + ] = False, + max_tasks: Annotated[ + Optional[int], + Field(description="Cap how many tasks are planned/executed; >= 1.", ge=1), + ] = None, + reference_dir: Annotated[ + Optional[str], + Field( + description=( + "Optional reference implementation to port from (analyzed " + "read-only). See the synchronous ``build`` tool." + ), + ), + ] = None, +) -> Dict[str, Any]: + """Start an autonomous build in the BACKGROUND and return immediately. + + Use when: the build may run for minutes and you want to keep working — this + returns a ``run_id`` right away instead of blocking (as the synchronous + ``build`` does) until the run finishes. Poll ``job_status`` with the + ``run_id`` to watch progress, ``stop_job`` to cancel, ``list_jobs`` to see + everything running. Do NOT use for a quick preview — use ``build`` with + ``dry_run=True``. Related: ``build`` (synchronous), ``report`` (final outcome). + + DESTRUCTIVE side effects (once running): edits files, makes git commits, and + calls an external LLM provider. Refuses to start a second job for a project + that already has one running (one writer per project). + + Returns ``{run_id, status}`` on success, or ``{error}`` when a job is already + running for this project. + """ + orch = ProjectOrchestrator() + + def _target(report) -> str: + parts = [goal, "--budget", str(budget)] + if parallel: + parts.append("--parallel") + if max_tasks is not None: + parts += ["--max-tasks", str(max_tasks)] + return orch.build( + path, " ".join(parts), reference_dir=reference_dir, progress_cb=report + ) + + try: + run_id = registry.start("build", path, _target, stop_hook=orch.request_stop) + except RuntimeError as e: + return {"error": str(e)} + return {"run_id": run_id, "status": "running"} + + +@mcp.tool( + annotations=ToolAnnotations( + title="Start a run of planned tasks in the background (returns a run_id)", + readOnlyHint=False, + destructiveHint=True, + idempotentHint=False, + openWorldHint=True, + ) +) +def run_async( + path: Annotated[ + str, + Field( + description=( + "Absolute path to a project with planned tasks (a devplan). " + "Example: '/Users/me/code/my-api'." + ), + examples=["/Users/me/code/my-api"], + min_length=1, + ), + ], +) -> Dict[str, Any]: + """Start executing a project's planned tasks in the BACKGROUND. + + Use when: a devplan exists and you want it executed without blocking — like + ``run`` but returns a ``run_id`` immediately. Poll ``job_status``, cancel + with ``stop_job``. Do NOT use to plan from a goal — that is ``build_async``. + Related: ``run`` (synchronous), ``build_async``. + + DESTRUCTIVE side effects (once running): edits files, makes git commits, and + calls an external LLM provider. Refuses a second job for a project that + already has one running. + + Returns ``{run_id, status}``, or ``{error}`` when one is already running. + """ + orch = ProjectOrchestrator() + + def _target(report) -> str: + orch.run_project(path, dry_run=False, progress_cb=report) + return f"Ran pending tasks for {path}." + + try: + run_id = registry.start("run", path, _target, stop_hook=orch.request_stop) + except RuntimeError as e: + return {"error": str(e)} + return {"run_id": run_id, "status": "running"} + + +@mcp.tool( + annotations=ToolAnnotations( + title="Check a background job's status", + readOnlyHint=True, + idempotentHint=True, + openWorldHint=False, + ) +) +def job_status( + run_id: Annotated[ + str, + Field( + description="The run_id returned by build_async or run_async.", + examples=["a1b2c3d4e5f6"], + min_length=1, + ), + ], +) -> Dict[str, Any]: + """Return a background job's current state. + + Use when: you started a job with ``build_async``/``run_async`` and want to + know whether it is still ``running`` or has ``succeeded``/``failed``/ + ``stopped`` — and, when finished, its report (``result``) or ``error``. + Read-only and idempotent. Related: ``list_jobs`` (all jobs), ``stop_job``. + + Returns the job object (run_id, kind, project_path, status, result, error, + timestamps), or ``{error}`` when the run_id is unknown. + """ + state = registry.status(run_id) + if state is None: + return {"error": f"unknown run_id: {run_id}"} + return state + + +@mcp.tool( + annotations=ToolAnnotations( + title="Stop a running background job", + readOnlyHint=False, + destructiveHint=False, + idempotentHint=True, + openWorldHint=False, + ) +) +def stop_job( + run_id: Annotated[ + str, + Field( + description="The run_id of the job to stop.", + examples=["a1b2c3d4e5f6"], + min_length=1, + ), + ], +) -> Dict[str, Any]: + """Request cooperative cancellation of a running background job. + + Use when: a ``build_async``/``run_async`` job should stop — it finishes any + in-flight task and starts no new work, then produces a partial report. Poll + ``job_status`` afterward to confirm it reaches ``stopped``. Idempotent: + stopping a finished or already-stopped job is a harmless no-op. Related: + ``job_status``, ``list_jobs``. + + Returns ``{run_id, stopping: true}`` when a running job was signalled, or + ``{run_id, stopping: false}`` when the id is unknown or already finished. + """ + return {"run_id": run_id, "stopping": registry.stop(run_id)} + + +@mcp.tool( + annotations=ToolAnnotations( + title="List all background jobs", + readOnlyHint=True, + idempotentHint=True, + openWorldHint=False, + ) +) +def list_jobs() -> Dict[str, Any]: + """List every background job this server has started and their states. + + Use when: you want an overview of all ``build_async``/``run_async`` jobs — + running and finished — e.g. to find a lost run_id. Read-only, idempotent. + Related: ``job_status`` (one job), ``stop_job``. + + Returns ``{jobs: [...]}``, each entry the same object ``job_status`` returns. + """ + return {"jobs": registry.list_jobs()} + + +@mcp.tool( + annotations=ToolAnnotations( + title="Propose a ranked plan of work for approval", + readOnlyHint=False, + destructiveHint=False, + idempotentHint=False, + openWorldHint=True, + ) +) +def propose_plan( + path: Annotated[ + str, + Field( + description=( + "Absolute path to the project to analyze. " + "Example: '/Users/me/code/my-api'." + ), + examples=["/Users/me/code/my-api"], + min_length=1, + ), + ], + budget: Annotated[ + float, + Field(description="Maximum US dollars to spend on analysis; > 0.", gt=0), + ] = _DEFAULT_MCP_BUDGET, +) -> Dict[str, Any]: + """Analyze the project and return ranked, UNAPPROVED work proposals. + + Use when: you want misterdev to recommend what to work on and let a human + approve a subset BEFORE any code is edited — the review gate. The proposals + are persisted, so ``get_plan`` re-reads them, ``approve_plan`` marks a + subset, and ``execute_plan`` builds the approved ones. The codebase is + analyzed in this process, so it never enters the client's context. Do NOT + use to execute immediately without review — that is ``build``. Related: + ``get_plan``, ``approve_plan``, ``execute_plan``. + + Side effects: spends LLM budget analyzing the project and writes the plan to + ``.orchestrator/proposed_plan.json``; it edits NO source code. + + Returns ``{items: [...]}`` — each item has an id, title, work_type, + rationale, and ``approved: false`` — or ``{error}`` on failure. + """ + return ProjectOrchestrator().propose_plan(path, f"--budget {budget}") + + +@mcp.tool( + annotations=ToolAnnotations( + title="Read the current proposed plan and its approval state", + readOnlyHint=True, + idempotentHint=True, + openWorldHint=False, + ) +) +def get_plan( + path: Annotated[ + str, + Field( + description="Absolute path to a project that has a proposed plan.", + examples=["/Users/me/code/my-api"], + min_length=1, + ), + ], +) -> Dict[str, Any]: + """Return the persisted proposed plan and which items are approved. + + Use when: you want to review the proposals from ``propose_plan`` (and see + what has been approved so far) before approving or executing. Read-only and + idempotent. Related: ``propose_plan``, ``approve_plan``, ``execute_plan``. + + Returns ``{items: [...]}`` (empty ``items`` when no plan has been proposed). + """ + plan = load_plan(path) + return {"items": plan or []} + + +@mcp.tool( + annotations=ToolAnnotations( + title="Approve or reject items in the proposed plan", + readOnlyHint=False, + destructiveHint=False, + idempotentHint=True, + openWorldHint=False, + ) +) +def approve_plan( + path: Annotated[ + str, + Field( + description="Absolute path to a project that has a proposed plan.", + examples=["/Users/me/code/my-api"], + min_length=1, + ), + ], + approve_all: Annotated[ + bool, + Field(description="Approve every item in the plan."), + ] = False, + approve_ids: Annotated[ + Optional[list[str]], + Field( + description="Ids to approve (as shown by get_plan, e.g. 'P-001').", + examples=[["P-001", "P-003"]], + ), + ] = None, + reject_ids: Annotated[ + Optional[list[str]], + Field( + description="Ids to un-approve. An id in both lists is rejected.", + examples=[["P-002"]], + ), + ] = None, +) -> Dict[str, Any]: + """Set the approval flags on the proposed plan, then persist. + + Use when: after ``propose_plan``/``get_plan`` you want to mark which items + should actually run. ``approve_all`` approves everything; otherwise + ``approve_ids`` are approved and ``reject_ids`` un-approved (reject wins a + tie — the safer default). Idempotent. Then call ``execute_plan``. Related: + ``propose_plan``, ``get_plan``, ``execute_plan``. + + Side effects: rewrites ``.orchestrator/proposed_plan.json``; edits no code. + + Returns ``{items: [...]}`` with updated flags, or ``{error}`` when no plan + exists to approve. + """ + plan = set_approval( + path, + item_ids=approve_ids, + approve_all=approve_all, + reject_ids=reject_ids, + ) + if plan is None: + return {"error": "no proposed plan for this project; call propose_plan first"} + return {"items": plan} + + +@mcp.tool( + annotations=ToolAnnotations( + title="Build the approved items from the proposed plan", + readOnlyHint=False, + destructiveHint=True, + idempotentHint=False, + openWorldHint=True, + ) +) +def execute_plan( + path: Annotated[ + str, + Field( + description="Absolute path to a project with an approved plan.", + examples=["/Users/me/code/my-api"], + min_length=1, + ), + ], + budget: Annotated[ + float, + Field(description="Maximum US dollars to spend; > 0.", gt=0), + ] = _DEFAULT_MCP_BUDGET, +) -> str: + """Execute the APPROVED items from a previously proposed plan. + + Use when: ``propose_plan`` + ``approve_plan`` have selected the work and you + want it built — this composes a goal from the approved items and runs the + normal build pipeline (decompose, edit, verify, revert regressions). Do NOT + use before approving anything (it returns a no-op message). Related: + ``propose_plan``, ``approve_plan``, ``build``. + + DESTRUCTIVE side effects: edits files, makes git commits, and calls an + external LLM provider. Refuses a dirty working tree (like ``build``). + + Returns a compact build report, or a message when nothing is approved. + """ + return ProjectOrchestrator().execute_plan(path, f"--budget {budget}") + + def main() -> None: """Console entry point: serve misterdev over stdio MCP.""" mcp.run() diff --git a/misterdev/task_executors/markdown_plan_executor/commands_mixin.py b/misterdev/task_executors/markdown_plan_executor/commands_mixin.py index ae8f0d5..d2597fd 100644 --- a/misterdev/task_executors/markdown_plan_executor/commands_mixin.py +++ b/misterdev/task_executors/markdown_plan_executor/commands_mixin.py @@ -4,7 +4,10 @@ from misterdev.core.models import Task from misterdev.core.execution.project import Project +from misterdev.core.execution.infra import infra_failure from misterdev.core.verification.validator import _run_cmd +from misterdev.agent_helpers import worktree_setup_command +from misterdev.config import get_setting from misterdev.utils.file_utils import write_file from .helpers import logger @@ -15,6 +18,58 @@ class CommandsMixin: # Command execution and file operations # ---------------------------------------------------------------- + def _run_gate( + self, project: Project, command: str, timeout: int, cwd=None + ) -> tuple: + """Run a build/typecheck/test gate command, self-healing an ENVIRONMENT + fault before trusting the failure. + + Runs ``command`` once. If it fails AND the output carries an + infrastructure signature (a timeout, a missing dependency, a locked store, + ENOSPC, OOM — see ``infra_failure``), the fault is in the worktree, not the + code: re-prime the worktree's dependencies and re-run the gate exactly + ONCE, returning that result. A plain code failure (a type error, a failed + assertion) has no infra signature, so it is returned as-is with no re-run. + Same timeout on both runs. Returns ``(success, output)``. + """ + success, output = self._run_command(project, command, timeout=timeout, cwd=cwd) + if success: + return success, output + reason = infra_failure(output) + if not reason: + return success, output + logger.warning( + "Gate failed on an environment fault (%s), not the code; re-priming " + "worktree dependencies and re-running once: %s", + reason, + command, + ) + self._reprime_worktree_deps(project) + return self._run_command(project, command, timeout=timeout, cwd=cwd) + + def _reprime_worktree_deps(self, project: Project) -> None: + """Re-run the project's dependency-priming command in the worktree root. + + Best-effort: no configured/detected setup command is a no-op, and a failed + re-prime only logs — the gate re-runs regardless so a genuine code failure + still surfaces. Runs at ``project.path`` (the worktree root, where the + lockfile lives), matching the priming done at worktree creation. + """ + setup_cmd = worktree_setup_command(project.config, project.path) + if not setup_cmd: + return + setup_timeout = get_setting( + project.config, "orchestrator", "worktree_setup_timeout" + ) + logger.info(f"Re-priming worktree dependencies: {setup_cmd}") + ok, out = self._run_command( + project, setup_cmd, timeout=setup_timeout, cwd=project.path + ) + if not ok: + logger.warning( + f"Dependency re-prime failed (re-running gate anyway): {out[-200:]}" + ) + def _run_command( self, project: Project, command: str, timeout: int = 120, cwd=None ) -> tuple: diff --git a/misterdev/task_executors/markdown_plan_executor/context_mixin.py b/misterdev/task_executors/markdown_plan_executor/context_mixin.py index b8bba4a..8dd7b13 100644 --- a/misterdev/task_executors/markdown_plan_executor/context_mixin.py +++ b/misterdev/task_executors/markdown_plan_executor/context_mixin.py @@ -222,6 +222,32 @@ def _solved_task_priors(self, project: Project, task: Task) -> str: logger.debug(f"Solved-task warm-start skipped: {e}") return "" + def _failure_priors(self, project: Project, task: Task) -> str: + """Runtime read-back of this project's FailureLog for THIS task, or "". + + The FailureLog was write-only at runtime (read only by the evolution loop); + this closes the loop within a run: a task that already failed sees its own + prior failures (by task id) so a later attempt does not rediscover the same + error. Best-effort — a missing/unreadable log or no match degrades to "". + """ + try: + from misterdev.core.learning.failure_log import FailureLog + + records = FailureLog( + project.path / ".orchestrator" / "failures.jsonl" + ).load() + mine = [r for r in records if r.name == task.id and r.error] + if not mine: + return "" + lines = ["## Prior failures of this task (do NOT repeat these):"] + for r in mine[::-1][:3]: # most recent first, capped + cat = f"[{r.category}] " if r.category else "" + lines.append(f"- {cat}{r.error.strip()[:300]}") + return "\n".join(lines) + except Exception as e: # read-back is best-effort; never sink the build + logger.debug(f"Failure-prior read-back skipped: {e}") + return "" + def _localize_target_files(self, project: Project, task: Task) -> List[str]: """Find edit targets for a task that declares none, or []. diff --git a/misterdev/task_executors/markdown_plan_executor/edits_mixin.py b/misterdev/task_executors/markdown_plan_executor/edits_mixin.py index 3264e77..734a399 100644 --- a/misterdev/task_executors/markdown_plan_executor/edits_mixin.py +++ b/misterdev/task_executors/markdown_plan_executor/edits_mixin.py @@ -281,10 +281,12 @@ def _runner(cmd, timeout): return False, "" return p.returncode == 0, (p.stdout or "") + (p.stderr or "") - # Score the largest edited non-test source file with a mutable changed - # region (the fix's main target). A SKIP (no mutable change, e.g. a - # whitespace-only or comment edit) falls through to the next candidate - # rather than aborting — the real fix may be in a smaller file. + # Score EVERY edited non-test source file with a mutable changed region, + # not just the largest: a multi-file fix's crux may be a small file, and + # a weak suite there is exactly what a single-file check misses. A SKIP + # (no mutable change, e.g. a whitespace-only or comment edit) is passed + # over; largest-first only orders the logs. + scored = 0 for path in sorted(pre_edit, key=lambda p: len(pre_edit[p]), reverse=True): if _is_test_file(path) or not (project.path / path).exists(): continue @@ -301,18 +303,19 @@ def _runner(cmd, timeout): ) if res.status == SKIP: continue + scored += 1 logger.info(f"Changed-region mutation [{path}]: {res.reason}") if res.status == RED: logger.warning( f"Weak suite for {path}: the passing test barely constrains " f"the fix — {res.reason}" ) - return # Enabled but nothing scorable: keep the seam observable, never silent. - logger.info( - "Changed-region mutation: no mutable source change to score " - f"(files: {sorted(pre_edit)})" - ) + if scored == 0: + logger.info( + "Changed-region mutation: no mutable source change to score " + f"(files: {sorted(pre_edit)})" + ) except Exception as e: # advisory only — never fail a completed task logger.debug(f"Changed-region mutation check skipped: {e}") diff --git a/misterdev/task_executors/markdown_plan_executor/execute_mixin.py b/misterdev/task_executors/markdown_plan_executor/execute_mixin.py index 0bd6952..bcf0442 100644 --- a/misterdev/task_executors/markdown_plan_executor/execute_mixin.py +++ b/misterdev/task_executors/markdown_plan_executor/execute_mixin.py @@ -13,6 +13,10 @@ from misterdev.core.execution.error_classifier import ( format_classified_error, ) +from misterdev.core.execution.escalation import ( + choose_rung, + should_count_failure, +) from misterdev.llm.client import CACHE_BREAKPOINT, SYSTEM_CACHE_SPLIT from misterdev.llm.prompt_manager import PromptManager from misterdev.config import get_setting @@ -271,6 +275,11 @@ def execute( # matches. Closes the learning loop at the executor, not just the planner. solved_priors = self._solved_task_priors(project, task) + # Runtime read-back of this task's own prior failures (T5.1): a later attempt + # sees what already went wrong this run instead of rediscovering it. "" when + # the FailureLog has nothing for this task. + failure_priors = self._failure_priors(project, task) + # Shared task-list preamble: the global conventions a numbered devplan # states once up front (canonical constants, locked dependency versions, # "never guess" rules) that every task must honor. The tasklist parser @@ -303,11 +312,45 @@ def execute( attempt_cap = max_retries no_output_forgiven = 0 forgiveness_cap = 2 + # Escalation ladder: count only NON-infra (real code) gate failures, and + # climb widen_context -> stronger_model -> decompose as they accumulate. + # Infra faults self-heal and must never advance the ladder. Off -> the + # counter stays 0 and every attempt is the plain "normal" rung. + escalation_on = get_setting( + project.config, "orchestrator", "escalation_enabled" + ) + code_failures = 0 while True: attempt += 1 if attempt >= attempt_cap: break logger.info(f"Attempt {attempt + 1}/{attempt_cap} for task {task.id}") + # Pick this attempt's rung from the code-failure count so far. On the + # decompose rung, stop retrying the whole task and request a split into + # named sub-steps instead of burning the remaining attempts. + rung = ( + self._escalation_rung(project, code_failures) + if escalation_on + else "normal" + ) + if rung == "decompose": + logger.info( + f"Task {task.id} escalated to DECOMPOSE after {code_failures} " + "code failure(s); requesting a split into sub-steps." + ) + self._abort_task( + project, branch_name, base_branch, snapshot, untracked_before + ) + return self._escalate_decompose( + project, + task, + error_logs, + use_git_branch=use_git_branch, + _depth=_depth, + ) + widen_context = rung in ("widen_context", "full_rewrite", "stronger_model") + force_stronger = rung == "stronger_model" + force_full_rewrite = rung == "full_rewrite" # Diagnostic: sizes of the context that ACCUMULATES across attempts, # so growth (or a runaway component) is visible per retry. logger.debug( @@ -384,10 +427,14 @@ def execute( # Budget-aware context allocation using configured token limit budget = ContextBudget(max_tokens=context_budget_tokens) - budget.set("code_context", code_context, priority=1) + # The edit region: the exact lines a SEARCH/REPLACE must match, kept + # verbatim so the model never edits blind against a dropped tail. + budget.set("code_context", code_context, priority=1, truncatable=False) # Correctness-critical: the complete reference set must survive # truncation, or a rename/delete misses sites and the build fails. - budget.set("reference_sites", reference_sites, priority=1, min_lines=0) + budget.set( + "reference_sites", reference_sites, priority=1, truncatable=False + ) # The self-authored reproduction test IS the task's concrete target; # it must survive truncation so the model always edits toward it. budget.set("spec_test", spec_test_source or "", priority=1, min_lines=0) @@ -401,6 +448,7 @@ def execute( budget.set("error_logs", error_logs or "", priority=1, min_lines=20) budget.set("scratchpad", scratchpad_context, priority=3) budget.set("solved_priors", solved_priors, priority=3, min_lines=0) + budget.set("failure_priors", failure_priors, priority=2, min_lines=0) # The user's own directive for this task must survive truncation. budget.set("user_answer", user_answer, priority=1, min_lines=0) budget.set("interface_contracts", interface_contracts, priority=2) @@ -441,6 +489,8 @@ def execute( full_code_context += "\n\n" + allocated["recent_changes"] if allocated["solved_priors"]: full_code_context += "\n\n" + allocated["solved_priors"] + if allocated["failure_priors"]: + full_code_context += "\n\n" + allocated["failure_priors"] if allocated["user_answer"]: full_code_context += ( "\n\n## The user's answer to your earlier question (follow this)\n" @@ -455,6 +505,16 @@ def execute( full_code_context += mcp_gathered if runtime_tool_ctx: full_code_context += runtime_tool_ctx + # Escalation: at the widen rung, re-anchor the model on the FULL, + # verbatim task spec (description + acceptance) and the exact target + # files and their dependents, so a fix that kept missing the point + # under a truncated/partial view sees the whole picture. + if widen_context: + full_code_context += self._escalation_spec_block( + project, task, target_files + ) + if force_full_rewrite: + full_code_context += self._full_rewrite_directive(target_files) full_code_context += self._mcp_awareness(project) guidance_context = " ".join( @@ -542,6 +602,19 @@ def execute( routed_model = self._select_model( project, task, strategy, attempt, attempt_cap ) + # Escalation: at the stronger-model rung, override the routed model + # with the configured stronger one (if any) — the cheap/default model + # has failed on real code repeatedly, so pay for a more capable one. + if force_stronger: + stronger = get_setting( + project.config, "orchestrator", "escalation_model" + ) + if stronger: + logger.info( + f"Escalation: routing {task.id} to stronger model " + f"{stronger} after {code_failures} code failure(s)." + ) + routed_model = stronger try: with project.llm_client.track_task(task.id): llm_response, aborted, pending_attempt = self._invoke_routed( @@ -670,7 +743,24 @@ def execute( if stall_risk > 0.7: logger.warning(f"High stall risk detected ({stall_risk:.2f}).") if attempt > 1: + # A stall — the model reproduced the same edit — is a + # genuine failure to make progress, so it advances the + # escalation ladder just like a red gate. Otherwise a + # staller would only ever see "try a different approach" + # and spin until attempts run out, never widening context + # / switching model / decomposing (the fix T005 needed). + code_failures += 1 error_logs = "ERROR: Stalling detected. Try a fundamentally different approach." + # Reset to the clean task base so the next attempt is a + # FRESH candidate, not another edit piled onto the stuck one + # the model keeps failing to fix (T3.2). Stays on the branch. + self._reset_to_task_base( + project, + branch_name, + base_branch, + snapshot, + untracked_before, + ) continue validation_failed = False @@ -761,13 +851,15 @@ def execute( gate_verified = False build_cmd = task_build_command if build_cmd: - success, output = self._run_command( - project, build_cmd, timeout=build_timeout, cwd=task_cwd + success, output = self._run_gate( + project, build_cmd, build_timeout, task_cwd ) if not success: logger.warning(f"Build failed on attempt {attempt + 1}") + if should_count_failure(output): + code_failures += 1 locations = resolver.resolve_errors(output) - attributed_error = resolver.format_for_llm(locations) + attributed_error = resolver.format_for_llm(locations, output) classified = format_classified_error(output) error_logs = self._build_error_context( prior_errors, @@ -784,13 +876,15 @@ def execute( gate_verified = True if typecheck_command: - success, output = self._run_command( - project, typecheck_command, timeout=build_timeout, cwd=task_cwd + success, output = self._run_gate( + project, typecheck_command, build_timeout, task_cwd ) if not success: logger.warning(f"Type check failed on attempt {attempt + 1}") + if should_count_failure(output): + code_failures += 1 locations = resolver.resolve_errors(output) - attributed_error = resolver.format_for_llm(locations) + attributed_error = resolver.format_for_llm(locations, output) classified = format_classified_error(output) error_logs = self._build_error_context( prior_errors, @@ -807,8 +901,8 @@ def execute( gate_verified = True if test_command: - success, output = self._run_command( - project, test_command, timeout=test_timeout, cwd=task_cwd + success, output = self._run_gate( + project, test_command, test_timeout, task_cwd ) # Whether the command genuinely exited zero on THIS tree — kept # separate from a flake rescue below so the acceptance short-circuit @@ -847,11 +941,14 @@ def execute( # baseline run OR a flake-rescued one must still be # re-checked, so key on the raw result, not the flake flip. already_passed=test_command if test_exited_green else None, + prior_gate_passed=True, ) if not acc_ok: logger.warning( f"Acceptance criteria not met on attempt {attempt + 1}." ) + if should_count_failure(acc_output): + code_failures += 1 classified = format_classified_error(acc_output) error_logs = self._build_acceptance_error_context( prior_errors, attempt, task, classified @@ -914,8 +1011,10 @@ def execute( ) else: logger.warning(f"Tests failed on attempt {attempt + 1}.") + if should_count_failure(output): + code_failures += 1 locations = resolver.resolve_errors(output) - attributed_error = resolver.format_for_llm(locations) + attributed_error = resolver.format_for_llm(locations, output) classified = format_classified_error(output) error_logs = self._build_error_context( prior_errors, @@ -953,11 +1052,14 @@ def execute( llm_acceptance_judge, test_timeout, cwd=task_cwd, + prior_gate_passed=gate_verified, ) if not acc_ok: logger.warning( f"Acceptance criteria not met on attempt {attempt + 1}." ) + if should_count_failure(acc_output): + code_failures += 1 classified = format_classified_error(acc_output) error_logs = self._build_acceptance_error_context( prior_errors, attempt, task, classified @@ -1000,6 +1102,31 @@ def execute( self._ledger_record(project, task, pending_attempt, success=False) pending_attempt = None + # Escalation ladder top rung, reached by EXHAUSTION. The in-loop decompose + # check can never fire when escalation_decompose_after >= the attempt cap + # (the default 3 == 3), because the loop breaks the moment code_failures + # reaches the threshold — so a keystone that failed decompose_after times + # would dead-end at a vague park instead of decomposing. Fire it here so + # the rung is reachable: split the too-large task into named sub-steps + # (the build pipeline re-decomposes them; a run --tasks park at least + # surfaces concrete steps) rather than burn a surgical retry that won't + # crack a structural failure. Only when the ladder genuinely climbed to + # decompose; otherwise fall through to the surgical retry unchanged. + if ( + escalation_on + and _depth < 1 + and not task.processor_data.get("_escalation_decomposed") + and self._escalation_rung(project, code_failures) == "decompose" + ): + logger.info( + f"Task {task.id} exhausted attempts at the decompose rung " + f"({code_failures} code failure(s)); requesting a split into sub-steps." + ) + self._abort_task( + project, branch_name, base_branch, snapshot, untracked_before + ) + return self._escalate_decompose(project, task, error_logs) + # Strategy escalation: if current strategy failed, try one more attempt # with "surgical". Guarded by _depth so escalation can never recurse more # than once, even if the strategy-selection logic changes later. @@ -1054,3 +1181,153 @@ def execute( f"Task failed after {max_retries} attempts + escalation.", error_logs, ) + + def _escalation_rung(self, project: Project, code_failures: int) -> str: + """The escalation rung for the next attempt, from the config thresholds.""" + return choose_rung( + code_failures, + widen_after=get_setting( + project.config, "orchestrator", "escalation_widen_after" + ), + rewrite_after=get_setting( + project.config, "orchestrator", "escalation_rewrite_after" + ), + model_after=get_setting( + project.config, "orchestrator", "escalation_model_after" + ), + decompose_after=get_setting( + project.config, "orchestrator", "escalation_decompose_after" + ), + ) + + def _escalation_spec_block(self, project: Project, task: Task, target_files) -> str: + """The verbatim task spec injected at the widen rung. + + A repeatedly-failing attempt was likely editing against a truncated view + or drifting from the goal, so re-anchor it on the FULL objective and + acceptance criteria plus the exact target files (shown in full above).""" + parts = ["\n\n## Escalation — full task spec (re-read; do not deviate)"] + if task.description: + parts.append(f"### Objective\n{task.description}") + criteria = (getattr(task, "acceptance_criteria", "") or "").strip() + if criteria: + parts.append(f"### Acceptance criteria (must be satisfied)\n{criteria}") + if target_files: + listed = "\n".join(f"- {f}" for f in target_files) + parts.append( + "### Target files (edit ONLY these; they are shown in full above " + f"and their call sites are listed)\n{listed}" + ) + return "\n\n".join(parts) + + @staticmethod + def _full_rewrite_directive(target_files) -> str: + """The directive injected at the full_rewrite rung — a structurally + different attempt. + + Incremental SEARCH/REPLACE patching has failed repeatedly (the model keeps + mis-anchoring against the view). Instruct it to abandon patching and emit + the COMPLETE new contents of the target file(s) from scratch, so the next + attempt is a different kind of edit rather than another reprompt.""" + listed = "\n".join(f"- {f}" for f in (target_files or [])) + return ( + "\n\n## Escalation — FULL REWRITE (incremental patches keep failing)\n" + "Stop patching. Do NOT produce small SEARCH/REPLACE edits — they have " + "failed to land repeatedly. Instead, rewrite the COMPLETE, final " + "contents of each target file below from scratch (the whole file), " + "correct and self-consistent, satisfying the acceptance criteria:\n" + f"{listed}" + ) + + @staticmethod + def _decompose_substeps(task: Task) -> list: + """Named sub-steps to split a stuck task into. Prefers the task's own + acceptance criteria (each line becomes a sub-step); falls back to a generic + isolate-then-extend split when there is nothing structured to lean on.""" + criteria = (getattr(task, "acceptance_criteria", "") or "").strip() + items = [] + for line in criteria.splitlines(): + s = line.strip().lstrip("-*0123456789.) ").strip() + if len(s) > 8: + items.append(f"Implement and verify: {s}") + if len(items) >= 2: + return items[:5] + label = task.title or (task.description or "")[:60] or task.id + return [ + f"Isolate the smallest failing part of '{label}' and make it pass alone", + f"Wire the remaining behavior of '{label}' on top, keeping the suite green", + ] + + def _build_substep_tasks(self, task: Task, substeps: list) -> list: + """Real child Tasks for a decomposed task's sub-steps. + + Each child carries one sub-step as both its description AND its acceptance + criterion (so it runs the full per-task gate), scoped to the parent's files, + and flagged as an escalation sub-step so it is never itself re-decomposed.""" + # The task reaching the decompose rung may be a duck object, so read every + # optional field defensively. + creates = list(getattr(task, "files_to_create", []) or []) + modifies = list(getattr(task, "files_to_modify", []) or []) + ctx = list(getattr(task, "context_files", []) or []) + children = [] + for i, step in enumerate(substeps): + children.append( + Task( + id=f"{task.id}-sub{i + 1}", + title=step[:80], + description=step, + acceptance_criteria=step, + files_to_create=creates, + files_to_modify=modifies, + context_files=ctx, + complexity=getattr(task, "complexity", "medium"), + category=getattr(task, "category", "feature"), + type=getattr(task, "type", "markdown_planner"), + status="pending", + project_ref=getattr(task, "project_ref", "."), + processor_data={"_is_escalation_substep": True, "parent": task.id}, + ) + ) + return children + + def _escalate_decompose( + self, + project: Project, + task: Task, + error_logs, + *, + use_git_branch: bool = True, + _depth: int = 0, + ): + """Top escalation rung: split the too-large task into named sub-steps and + EXECUTE each as a real child task (running the full per-task gate pipeline). + + Guarded by ``_depth``: a sub-step runs at depth+1 and can never itself + re-decompose, so recursion is bounded to one level. If every sub-step + completes, the parent is satisfied; otherwise (or at depth >= 1) the task is + parked (deferred) with the decomposition request as before. The caller has + already reverted this task's work, so the children start from a clean base. + """ + substeps = self._decompose_substeps(task) + task.processor_data["_escalation_decomposed"] = True + task.processor_data["escalation_substeps"] = substeps + if _depth < 1 and substeps: + results = [ + self.execute( + child, project, use_git_branch=use_git_branch, _depth=_depth + 1 + ) + for child in self._build_substep_tasks(task, substeps) + ] + if results and all(r.status == "completed" for r in results): + return self._complete_task( + project, + task, + f"Completed via {len(results)} executed sub-step(s).", + error_logs or "", + ) + reason = ( + f"escalated to decomposition: '{task.title or task.id}' failed " + "repeatedly and is too large to land in one edit. Split it into the " + "sub-steps below and run them." + ) + return self._defer_task(project, task, reason, substeps, error_logs) diff --git a/misterdev/task_executors/markdown_plan_executor/gates_mixin.py b/misterdev/task_executors/markdown_plan_executor/gates_mixin.py index a9cd5c5..1295f7e 100644 --- a/misterdev/task_executors/markdown_plan_executor/gates_mixin.py +++ b/misterdev/task_executors/markdown_plan_executor/gates_mixin.py @@ -211,13 +211,22 @@ def _gate_accepts( the whole suite green. An unparseable red result stays strict (rejected), since we will not accept on a number we cannot read. """ + from misterdev.core.verification.validator import ( + _parse_test_counts, + gate_ran_no_tests, + ) + if success: + # A command that exits 0 having collected nothing is a false-GREEN + # gate: it greenlights any edit while catching no regression. Pair an + # explicit "no tests ran" signal with a parsed total of 0 (high + # precision — the phrase alone can appear per-crate in a healthy + # workspace) and reject it as hard as a real failure. + if gate_ran_no_tests(output) and _parse_test_counts(output)[0] == 0: + return False, None return True, 0 if baseline_failures <= 0: return False, None - from misterdev.core.verification.validator import ( - _parse_test_counts, - ) total, post = _parse_test_counts(output) if total > 0 and post <= baseline_failures: @@ -234,15 +243,28 @@ def _confirm_flaky( a misleading error and, on final failure, reverts a correct task. Re-runs the SAME command on the SAME (unchanged) tree, so a non-reproducing failure is nondeterministic by construction and must not block. ``reruns`` <= 0 - disables it (the default), preserving the strict single-run gate.""" - if reruns <= 0: + disables it (the default), preserving the strict single-run gate — EXCEPT + when the failure carries an environment/infra signature (a timeout, a + missing dependency, a locked store), where a self-healing re-run is always + warranted because the fault is not in the code.""" + from misterdev.core.execution.infra import infra_failure + + infra = infra_failure(output) + effective_reruns = reruns if reruns > 0 else (1 if infra else 0) + if effective_reruns <= 0: return False + if infra: + logger.warning( + "Test gate failed on an environment fault (%s), not the code; " + "re-running once before trusting it.", + infra, + ) from misterdev.core.verification.flaky import confirm_test_failure def _rerun(): return self._run_command(project, test_command, timeout=timeout, cwd=cwd) - verdict = confirm_test_failure(_rerun, output, reruns) + verdict = confirm_test_failure(_rerun, output, effective_reruns) if not verdict.is_real_failure: logger.warning( "Per-task test failure did not reproduce (%s); treating as a flake, " @@ -261,6 +283,7 @@ def _verify_acceptance( timeout: int, cwd=None, already_passed: Optional[str] = None, + prior_gate_passed: bool = True, ) -> Tuple[bool, str]: """Verify the task's acceptance_criteria after build/test gates pass. @@ -306,7 +329,7 @@ def _verify_acceptance( # false-failed on `cargo test` from the repo root). Treat it as a # pass-through. A genuine missing test path (FILE_NOT_FOUND) is left # as a real failure. - if classify_error(output) == ErrorCategory.MANIFEST: + if prior_gate_passed and classify_error(output) == ErrorCategory.MANIFEST: logger.warning( "Acceptance command could not locate the project manifest; " "the build/test gates already passed, so treating acceptance " @@ -347,11 +370,14 @@ def _judge_generate(self, project: Project, prompt: str) -> str: on the generator's own model. Routed through ``with_model`` when possible. """ from misterdev.core.verification.independent import ( - generate_independent, + build_independent_call, ) judge_model = (project.config.get("judge") or {}).get("model") - return generate_independent(project.llm_client, prompt, "", model=judge_model) + call = build_independent_call( + project.llm_client, "", judge_model, "Acceptance judge" + ) + return call(prompt) if call is not None else "" def _llm_acceptance_judge( self, project: Project, task: Task, criteria: str diff --git a/misterdev/task_executors/markdown_plan_executor/git_mixin.py b/misterdev/task_executors/markdown_plan_executor/git_mixin.py index 4e3d4f8..fb14bec 100644 --- a/misterdev/task_executors/markdown_plan_executor/git_mixin.py +++ b/misterdev/task_executors/markdown_plan_executor/git_mixin.py @@ -201,6 +201,34 @@ def _abort_task( if topo is not None and hasattr(topo, "invalidate"): topo.invalidate() + def _reset_to_task_base( + self, + project: Project, + branch_name: Optional[str], + base_branch: Optional[str], + snapshot: Optional[Dict], + untracked_before: Optional[set] = None, + ) -> None: + """Discard the current stuck candidate and return the tree to the task's + clean base WITHOUT leaving the branch or the retry loop. + + Unlike :meth:`_abort_task` (which checks out the base and deletes the + branch to end the task), this stays ON the task branch so the loop can make + a FRESH attempt that re-derives from the clean base — the T3.2 stall reset — + instead of piling the next edit onto edits the model keeps failing to fix. + Task edits during the loop are uncommitted, so ``git reset --hard`` drops + them back to base; untracked orphans the stuck attempt wrote are cleaned. + """ + if branch_name and base_branch: + self._git(project, "git reset --hard") + self._clean_task_orphans(project, untracked_before) + elif snapshot is not None: + self._revert_files(project, snapshot) + self._clean_task_orphans(project, untracked_before) + topo = getattr(project, "topography", None) + if topo is not None and hasattr(topo, "invalidate"): + topo.invalidate() + def _clean_task_orphans( self, project: Project, untracked_before: Optional[set] ) -> None: diff --git a/misterdev/task_executors/markdown_plan_executor/results_mixin.py b/misterdev/task_executors/markdown_plan_executor/results_mixin.py index c2e3ba7..c50346f 100644 --- a/misterdev/task_executors/markdown_plan_executor/results_mixin.py +++ b/misterdev/task_executors/markdown_plan_executor/results_mixin.py @@ -89,9 +89,14 @@ def _deferral_reason(self, task: Task, error_logs: str, has_gate: bool): "effort — please review and confirm it is right, or tell me what " "to change.", ) - tail = "" - if error_logs and error_logs.strip(): - tail = error_logs.strip().splitlines()[-1][:160] + # The last MEANINGFUL error line — skipping blank lines and markdown code + # fences, which otherwise made the parked question read "last error: ```". + meaningful = [ + ln.strip() + for ln in (error_logs or "").splitlines() + if ln.strip() and not ln.strip().startswith("```") + ] + tail = meaningful[-1][:200] if meaningful else "" return ( "could not complete after all attempts", f"I couldn't finish '{title}'" diff --git a/misterdev/tools/git_tool.py b/misterdev/tools/git_tool.py index cdd03de..80e3bb9 100644 --- a/misterdev/tools/git_tool.py +++ b/misterdev/tools/git_tool.py @@ -54,13 +54,27 @@ def worktree_remove(self, project: Any, path: str) -> Tuple[bool, str]: project, command=f"git worktree remove --force {shlex.quote(path)}" ) + def worktree_prune(self, project: Any) -> Tuple[bool, str]: + """Drop administrative metadata for worktrees whose directory is gone + (e.g. a prior run crashed mid-task), so a fresh run starts clean.""" + return super().execute(project, command="git worktree prune") + def merge_worktree(self, project: Any, branch: str) -> Tuple[bool, str]: - """Merge a worktree's branch into the current branch, then delete it.""" + """Merge a worktree's branch into the current branch, then delete it. + + A conflicting or otherwise failed merge leaves the tree mid-merge + (``MERGE_HEAD`` set, conflict markers written), which would block the next + merge and leave the base branch dirty. So on failure we ``git merge + --abort`` to restore the pre-merge base cleanly and never force-merge — the + caller re-queues the task instead. The abort is best-effort (a no-op if no + merge was actually started).""" success, out = super().execute( project, command=f"git merge --no-ff {shlex.quote(branch)} --no-edit" ) if success: super().execute(project, command=f"git branch -d {shlex.quote(branch)}") + else: + super().execute(project, command="git merge --abort") return success, out def branch_create(self, project: Any, branch: str) -> Tuple[bool, str]: @@ -77,6 +91,14 @@ def merge(self, project: Any, branch: str) -> Tuple[bool, str]: def checkout(self, project: Any, branch: str) -> Tuple[bool, str]: return super().execute(project, command=f"git checkout {shlex.quote(branch)}") + def reset_hard(self, project: Any, ref: str = "HEAD") -> Tuple[bool, str]: + """Hard-reset the current branch to ``ref`` (drops the commits after it). + + Used to roll back a wave merge that broke the base branch: the merge is + ``--no-ff`` so it is the tip, and ``HEAD^`` is the pre-merge base tip. + """ + return super().execute(project, command=f"git reset --hard {shlex.quote(ref)}") + def status(self, project: Any) -> Tuple[bool, str]: return super().execute(project, command="git status") diff --git a/tests/test_acceptance_manifest_passthrough_requires_prior_gate.py b/tests/test_acceptance_manifest_passthrough_requires_prior_gate.py new file mode 100644 index 0000000..cea6868 --- /dev/null +++ b/tests/test_acceptance_manifest_passthrough_requires_prior_gate.py @@ -0,0 +1,60 @@ +"""T0.1 — the MANIFEST acceptance-command pass-through requires a prior gate. + +`_verify_acceptance` treats a broken acceptance command that errors with a +MANIFEST/CONFIG classification as "satisfied", on the premise that acceptance runs +only after the build/test gates pass, so the manifest demonstrably exists. That +premise holds on the tests-passed path but is FALSE on the certainty-merge path, +where acceptance runs with no objective gate. There, a malformed acceptance command +must not silently pass — that is a false completion resting on the LLM's word plus a +broken command. +""" + +import types + +from misterdev.task_executors.markdown_plan_executor.gates_mixin import GatesMixin + +# Classifies as ErrorCategory.MANIFEST (verified against error_classifier). +MANIFEST_ERR = "error: could not find `Cargo.toml` in /x or any parent directory" + + +class _Stub(GatesMixin): + def __init__(self, output): + self._output = output + + def _run_command(self, project, command, timeout=None, cwd=None): + return (False, self._output) + + +def _task(criteria="Run `cargo test` to verify."): + return types.SimpleNamespace(acceptance_criteria=criteria, description="t", id="x") + + +def test_manifest_passthrough_denied_without_prior_gate(): + stub = _Stub(MANIFEST_ERR) + ok, _out = stub._verify_acceptance( + None, _task(), True, False, 30, prior_gate_passed=False + ) + assert ok is False, ( + "a broken (MANIFEST) acceptance command must NOT be treated as satisfied " + "when no objective gate ran (the certainty-merge path)" + ) + + +def test_manifest_passthrough_allowed_after_prior_gate(): + # Control: on the tests/build-passed path the premise holds; keep the + # pass-through so a broken command does not false-fail a genuinely gated task. + stub = _Stub(MANIFEST_ERR) + ok, _out = stub._verify_acceptance( + None, _task(), True, False, 30, prior_gate_passed=True + ) + assert ok is True + + +def test_no_criteria_gateless_task_still_passes(): + # Control: a genuinely gate-less task with no acceptance criteria is unaffected + # (the deliberate certainty-only completion path must not regress). + stub = _Stub("") + ok, _ = stub._verify_acceptance( + None, _task(criteria=""), True, False, 30, prior_gate_passed=False + ) + assert ok is True diff --git a/tests/test_adaptive.py b/tests/test_adaptive.py new file mode 100644 index 0000000..56589b4 --- /dev/null +++ b/tests/test_adaptive.py @@ -0,0 +1,60 @@ +"""Adaptive concurrency/timeout backoff decision (pure function).""" + +from misterdev.core.execution.adaptive import WaveTuning, next_wave_tuning + + +def test_backoff_on_infra_above_threshold(): + """A wave over the threshold halves workers and grows the timeout factor.""" + nxt = next_wave_tuning(3, WaveTuning(8, 1.0), base_workers=8, threshold=1) + assert nxt == WaveTuning(4, 2.0) + + +def test_recover_on_clean_wave(): + """A clean wave (0 infra) steps workers and timeout back toward the base.""" + nxt = next_wave_tuning(0, WaveTuning(2, 4.0), base_workers=8) + assert nxt == WaveTuning(4, 2.0) + + +def test_hold_steady_at_or_below_threshold(): + """A nonzero count at/below the threshold neither backs off nor recovers.""" + cur = WaveTuning(4, 2.0) + assert next_wave_tuning(1, cur, base_workers=8, threshold=1) == cur + + +def test_worker_floor_respected(): + """Backoff never drops below the worker floor, even from the floor.""" + nxt = next_wave_tuning(9, WaveTuning(1, 2.0), base_workers=8, min_workers=1) + assert nxt.max_workers == 1 + + +def test_timeout_ceiling_respected(): + """The timeout factor is capped at max_timeout_factor under sustained infra.""" + nxt = next_wave_tuning( + 9, WaveTuning(4, 4.0), base_workers=8, max_timeout_factor=4.0 + ) + assert nxt.timeout_factor == 4.0 + + +def test_recovery_clamps_to_base_workers_and_factor_one(): + """Recovery never overshoots the configured base workers or drops below 1.0.""" + nxt = next_wave_tuning(0, WaveTuning(8, 1.0), base_workers=8) + assert nxt == WaveTuning(8, 1.0) + + +def test_backoff_then_recover_round_trip(): + """A bad wave then two clean waves returns to full concurrency and factor 1.0.""" + t = WaveTuning(8, 1.0) + t = next_wave_tuning(5, t, base_workers=8) # -> (4, 2.0) + assert t == WaveTuning(4, 2.0) + t = next_wave_tuning(0, t, base_workers=8) # -> (8, 1.0) + assert t == WaveTuning(8, 1.0) + + +def test_base_workers_one_stays_one(): + """With a configured cap of 1, workers stay pinned at 1 through backoff and + recovery; only the timeout factor adapts.""" + t = WaveTuning(1, 1.0) + t = next_wave_tuning(5, t, base_workers=1) + assert t.max_workers == 1 and t.timeout_factor == 2.0 + t = next_wave_tuning(0, t, base_workers=1) + assert t == WaveTuning(1, 1.0) diff --git a/tests/test_changed_region_mutation.py b/tests/test_changed_region_mutation.py index 4fbd2be..91d18af 100644 --- a/tests/test_changed_region_mutation.py +++ b/tests/test_changed_region_mutation.py @@ -171,3 +171,23 @@ def test_seam_falls_back_to_project_test_command(tmp_path, caplog, monkeypatch): ex._changed_region_mutation_check(proj, {"mod.py": "orig"}, None, None) assert seen.get("cmd") == "python -m pytest -q" # used the project fallback assert any("Changed-region mutation [mod.py]" in r.message for r in caplog.records) + + +def test_changed_line_indices_unchanged_file_is_empty(): + from misterdev.core.verification.changed_region_mutation import changed_line_indices + + src = "def f():\n return 1\n" + assert changed_line_indices(src, src) == set() + + +def test_changed_line_indices_new_file_is_all_changed(): + from misterdev.core.verification.changed_region_mutation import changed_line_indices + + assert changed_line_indices("", "a\nb\nc") == {0, 1, 2} + + +def test_changed_line_indices_real_change_is_scoped(): + from misterdev.core.verification.changed_region_mutation import changed_line_indices + + idx = changed_line_indices("a\nb\nc", "a\nX\nc") + assert 1 in idx and 0 not in idx and 2 not in idx diff --git a/tests/test_compile_view_adapters.py b/tests/test_compile_view_adapters.py new file mode 100644 index 0000000..4599342 --- /dev/null +++ b/tests/test_compile_view_adapters.py @@ -0,0 +1,71 @@ +"""T2.1 — compile_view is a per-language adapter registry, not an if/else. + +Asserts the adapter interface + registry, a fully-working TypeScript (tsc) adapter +alongside the existing rust one, and the contract of the go/swift/csharp stubs +(registered, interface-conformant, recognizing nothing yet). Rust behavior is +covered by test_compile_view.py and must remain unaffected. +""" + +import pytest + +from misterdev.core.execution.compile_view import ( + CompilerAdapter, + extract_compile_errors, + get_adapter, + registered_languages, +) + +# Classic tsc output (two diagnostics) and the --pretty variant of the first. +TSC_CLASSIC = ( + "src/index.ts(12,7): error TS2322: Type 'string' is not assignable to type 'number'.\n" + "src/index.ts(20,3): error TS2554: Expected 1 arguments, but got 0.\n" +) +TSC_PRETTY = "src/index.ts:12:7 - error TS2322: Type 'string' is not assignable to type 'number'.\n" + +STUB_LANGUAGES = [] # go/swift/csharp promoted to real adapters (T2.1b) +FULL_LANGUAGES = ["rust", "typescript", "go", "swift", "csharp"] + + +def test_registry_has_all_five_languages(): + langs = registered_languages() + for lang in FULL_LANGUAGES + STUB_LANGUAGES: + assert lang in langs, f"{lang} adapter must be registered" + + +@pytest.mark.parametrize("lang", FULL_LANGUAGES + STUB_LANGUAGES) +def test_every_adapter_conforms_to_the_interface(lang): + ad = get_adapter(lang) + assert isinstance(ad, CompilerAdapter) + assert ad.language == lang + assert isinstance(ad.name, str) and ad.name + assert isinstance(ad.parse("some output"), list) + assert isinstance(ad.detect("some output"), bool) + + +def test_typescript_classic_extracted(): + errs = extract_compile_errors(TSC_CLASSIC, language="typescript") + by_code = {e.code: e for e in errs} + assert set(by_code) == {"TS2322", "TS2554"} + assert by_code["TS2322"].location == "src/index.ts:12:7" + assert "not assignable" in by_code["TS2322"].message + assert by_code["TS2554"].location == "src/index.ts:20:3" + + +def test_typescript_pretty_extracted(): + errs = extract_compile_errors(TSC_PRETTY, language="typescript") + assert any(e.code == "TS2322" and e.location == "src/index.ts:12:7" for e in errs) + + +def test_typescript_autodetected_without_language(): + errs = extract_compile_errors(TSC_CLASSIC) + assert any(e.code == "TS2322" for e in errs) + + +def test_typescript_does_not_misparse_rust(): + # A rust type error must not be claimed by the tsc adapter. + assert extract_compile_errors( + "error[E0308]: mismatched types", language="typescript" + ) == [] or all( + e.code and e.code.startswith("E") + for e in extract_compile_errors("error[E0308]: mismatched types") + ) diff --git a/tests/test_compile_view_go_swift_csharp.py b/tests/test_compile_view_go_swift_csharp.py new file mode 100644 index 0000000..c3b9a53 --- /dev/null +++ b/tests/test_compile_view_go_swift_csharp.py @@ -0,0 +1,78 @@ +"""T2.1b — real go / swift / csharp compile_view adapters (were stubs). + +Each parses verbatim compiler output into CompileError records (message, code, +location) and detects its own output for the no-language-hint fallback, at the same +altitude as the rust/tsc adapters. Cross-language detection must not bleed: a rust or +tsc diagnostic is never claimed by these parsers. +""" + +from misterdev.core.execution.compile_view import ( + extract_compile_errors, + get_adapter, +) + +# --- verbatim compiler output ------------------------------------------------- + +GO = ( + "# example/app\n" + "./main.go:10:6: undefined: helper\n" + "./util.go:4:2: cannot use n (variable of type int) as string value in argument\n" +) +SWIFT = ( + "/src/App.swift:12:15: error: cannot find 'foo' in scope\n" + "/src/App.swift:20:9: error: value of type 'Widget' has no member 'render'\n" +) +CSHARP = ( + "Program.cs(12,20): error CS0103: The name 'foo' does not exist in the current context\n" + "src/Svc.cs(30,13): error CS1002: ; expected [/repo/App.csproj]\n" +) + + +def test_go_extracted(): + errs = extract_compile_errors(GO, language="go") + locs = {e.location for e in errs} + assert "./main.go:10:6" in locs + assert any("undefined: helper" in e.message for e in errs) + assert "./util.go:4:2" in locs + + +def test_go_autodetected(): + assert any(e.location == "./main.go:10:6" for e in extract_compile_errors(GO)) + + +def test_swift_extracted(): + errs = extract_compile_errors(SWIFT, language="swift") + by_loc = {e.location: e for e in errs} + assert "/src/App.swift:12:15" in by_loc + assert "cannot find 'foo' in scope" in by_loc["/src/App.swift:12:15"].message + assert "/src/App.swift:20:9" in by_loc + + +def test_swift_autodetected(): + assert any( + e.location == "/src/App.swift:12:15" for e in extract_compile_errors(SWIFT) + ) + + +def test_csharp_extracted(): + errs = extract_compile_errors(CSHARP, language="csharp") + by_code = {e.code: e for e in errs} + assert set(by_code) == {"CS0103", "CS1002"} + assert by_code["CS0103"].location == "Program.cs:12:20" + # the trailing MSBuild [project] tag is stripped from the message + assert by_code["CS1002"].message == "; expected" + assert by_code["CS1002"].location == "src/Svc.cs:30:13" + + +def test_csharp_autodetected(): + assert any(e.code == "CS0103" for e in extract_compile_errors(CSHARP)) + + +def test_no_cross_language_detection_bleed(): + # rust and tsc outputs must not be claimed by go/swift/csharp adapters. + rust = "error[E0308]: mismatched types\n --> src/lib.rs:2:18" + tsc = "src/index.ts(12,7): error TS2322: Type 'string' is not assignable to type 'number'." + for lang in ("go", "swift", "csharp"): + ad = get_adapter(lang) + assert ad.detect(rust) is False + assert ad.detect(tsc) is False diff --git a/tests/test_decomposer.py b/tests/test_decomposer.py index bd8a49f..03836df 100644 --- a/tests/test_decomposer.py +++ b/tests/test_decomposer.py @@ -242,3 +242,29 @@ def generate_code(self, prompt, system=""): ) assert "clients/web/" in captured["prompt"] assert "ONE target" in captured["prompt"] + + +def test_cap_prunes_dangling_dependency_on_dropped_task(): + from misterdev.core.models import Task + from misterdev.core.planning.decomposer import _cap_tasks + + def _t(tid, deps=None): + return Task(id=tid, description="x", project_ref=".", dependencies=list(deps or [])) + + # C is the 3rd task -> dropped at cap 2; A's dependency on it must be pruned. + tasks = [_t("A", deps=["C"]), _t("B"), _t("C")] + out = _cap_tasks(tasks, 2) + assert [x.id for x in out] == ["A", "B"] + assert out[0].dependencies == [] + + +def test_cap_noop_under_limit_preserves_deps(): + from misterdev.core.models import Task + from misterdev.core.planning.decomposer import _cap_tasks + + tasks = [ + Task(id="A", description="x", project_ref=".", dependencies=["B"]), + Task(id="B", description="x", project_ref="."), + ] + assert _cap_tasks(tasks, 5) is tasks + assert tasks[0].dependencies == ["B"] diff --git a/tests/test_deferral.py b/tests/test_deferral.py index c95ab83..3fa87d8 100644 --- a/tests/test_deferral.py +++ b/tests/test_deferral.py @@ -37,6 +37,51 @@ def test_blocked_reason_ignores_real_code_errors(): assert blocked_reason(out) is None, out +def test_blocked_reason_not_fooled_by_key_feature_tests(): + # A task that IMPLEMENTS api-key issuance emits test text mentioning keys; a + # validation RESULT ("invalid") or a response-field assertion must NOT be + # misread as a missing external credential (the T018 false-positive). + for out in ( + "expected api key to be valid, got invalid", + "test: issued key hash invalid", + "invalid api key rejected as expected", + "apiKey missing from response body", + ): + assert blocked_reason(out) is None, out + # A genuine environment-absence still blocks. + for out in ( + "CLOUDFLARE_API_TOKEN is required", + "set the NPM_TOKEN env var", + "provide your API token", + ): + assert blocked_reason(out), out + + +def test_401_403_in_a_test_run_is_not_a_block(): + # A 401/403 emitted inside a test run (a @cloudflare/vitest-pool-workers probe, + # or a test asserting on a 401/403 response) is code to fix, not a credential + # block — the T007 keystone false-positive that froze the whole server subtree. + for out in ( + "FAIL src/index.test.ts\n@cloudflare/vitest-pool-workers\nError: 401 Unauthorized", + "expect(res.status).toBe(401)\n● app shell > rejects unauthenticated", + "vitest\n403 Forbidden returned by handler under test", + "describe('auth', () => it('returns 401')) # 401 unauthorized", + ): + assert blocked_reason(out) is None, out + + +def test_real_deploy_auth_still_blocks_outside_test_context(): + # A 401 / login instruction from a deploy or CLI (no test signatures) is a + # genuine external block and must still park; the SPECIFIC rules also fire even + # if a test happens to be mentioned. + assert blocked_reason( + "wrangler deploy --minify\nError: authentication failed (401)" + ) + assert blocked_reason("You are not logged in. Run `wrangler login`.") + # Non-suppressible signal wins even under test context. + assert blocked_reason("vitest run\nError: STRIPE_API_KEY is required") + + # --- NEEDS_INPUT model marker ------------------------------------------------- @@ -111,6 +156,40 @@ def test_deferral_reason_three_shapes(): assert "could not complete" in reason and "How should I proceed" in q +def test_deferral_reason_uses_meaningful_error_tail_not_fence(): + e = MarkdownPlanExecutor() + # error_logs ending in a code fence must not make the question say "```". + logs = "Type error in src/x.ts:\n```\nTS2345: argument type mismatch\n```\n" + _, q = e._deferral_reason(_task(), logs, has_gate=True) + assert "TS2345: argument type mismatch" in q and "```" not in q + + +def test_recover_to_base_branch_unstrands_head(tmp_path): + import subprocess + + from misterdev.agent import ProjectOrchestrator + + def git(*a): + subprocess.run(["git", *a], cwd=tmp_path, check=True, capture_output=True) + + git("init", "-b", "main") + git("config", "user.email", "t@t") + git("config", "user.name", "t") + (tmp_path / "a.txt").write_text("1") + git("add", "-A") + git("commit", "-m", "init") + git("checkout", "-b", "task/T005") # simulate a stranded task branch + project = SimpleNamespace(path=tmp_path) + ProjectOrchestrator()._recover_to_base_branch(project, "main") + head = subprocess.run( + ["git", "rev-parse", "--abbrev-ref", "HEAD"], + cwd=tmp_path, + capture_output=True, + text=True, + ).stdout.strip() + assert head == "main" + + def test_defer_task_returns_deferred_status_with_questions(): e = MarkdownPlanExecutor() statuses = [] @@ -233,3 +312,14 @@ def spy(ready, executor, proj): encoding="utf-8" ) assert all(t in progress for t in ("T1", "T2", "T3")) # all recorded completed + + +def test_load_answers_preserves_task_id_with_spaced_hyphen(tmp_path): + from misterdev.core.execution.deferral import DeferralBook + + q = tmp_path / ".orchestrator" / "QUESTIONS.md" + q.parent.mkdir(parents=True) + q.write_text("## auth - login — needs a secret\n\n- Answer: use env var\n") + book = DeferralBook(q.parent) + answers = book.load_answers() + assert answers.get("auth - login") == "use env var" diff --git a/tests/test_dep_clone.py b/tests/test_dep_clone.py new file mode 100644 index 0000000..3095fc7 --- /dev/null +++ b/tests/test_dep_clone.py @@ -0,0 +1,130 @@ +"""Copy-on-write cloning of a base checkout's dependency dirs into a worktree.""" + +import os +import tempfile +from pathlib import Path + +import pytest + +from misterdev.core.execution.dep_clone import ( + clone_dependencies, + clone_supported, + find_node_modules_dirs, +) + + +def _make_base(root: Path): + """A minimal monorepo: a root node_modules, a workspace package, and a server + whose node_modules links that package with a RELATIVE symlink (as pnpm does).""" + (root / "packages" / "shared").mkdir(parents=True) + (root / "packages" / "shared" / "index.js").write_text("export const x = 1\n") + (root / "node_modules").mkdir() + (root / "node_modules" / ".marker").write_text("root\n") + scope = root / "apps" / "server" / "node_modules" / "@scope" + scope.mkdir(parents=True) + (root / "apps" / "server" / "node_modules" / "realdep").mkdir() + (root / "apps" / "server" / "node_modules" / "realdep" / "index.js").write_text("1") + # Relative workspace symlink, exactly like pnpm's. + os.symlink("../../../../packages/shared", scope / "shared") + # Nested node_modules inside a package's node_modules must NOT be found + # separately (it rides along with the parent clone). + (root / "node_modules" / "dep" / "node_modules").mkdir(parents=True) + + +def test_find_node_modules_dirs_roots_and_workspaces(tmp_path): + _make_base(tmp_path) + found = {str(p) for p in find_node_modules_dirs(tmp_path)} + assert found == {"node_modules", os.path.join("apps", "server", "node_modules")} + # The nested node_modules inside node_modules/dep is pruned. + assert os.path.join("node_modules", "dep", "node_modules") not in found + + +def test_find_skips_git_and_orchestrator(tmp_path): + (tmp_path / ".git" / "node_modules").mkdir(parents=True) + (tmp_path / ".orchestrator" / "node_modules").mkdir(parents=True) + (tmp_path / "node_modules").mkdir() + assert [str(p) for p in find_node_modules_dirs(tmp_path)] == ["node_modules"] + + +def test_clone_supported_returns_bool(tmp_path): + assert isinstance(clone_supported(tmp_path), bool) + + +def test_clone_dependencies_no_node_modules_returns_false(tmp_path): + (tmp_path / "src").mkdir() + assert clone_dependencies(tmp_path, tmp_path / "wt") == (False, []) + + +@pytest.mark.skipif( + not clone_supported(tempfile.gettempdir()), + reason="filesystem does not support CoW/hardlink cloning", +) +def test_clone_preserves_relative_workspace_symlink_into_worktree(tmp_path): + """The cloned node_modules' relative workspace symlink resolves to the + WORKTREE's own package, not the base — the property that makes a cloned + worktree usable with no reinstall.""" + base = tmp_path / "base" + base.mkdir() + _make_base(base) + # A worktree has the SOURCE checked out (packages/shared) but no node_modules. + wt = tmp_path / "wt" + (wt / "packages" / "shared").mkdir(parents=True) + (wt / "packages" / "shared" / "index.js").write_text("export const x = 1\n") + (wt / "apps" / "server").mkdir(parents=True) + + ok, dirs = clone_dependencies(base, wt) + assert ok + assert set(dirs) == {"node_modules", os.path.join("apps", "server", "node_modules")} + + # Root clone carried its regular files. + assert (wt / "node_modules" / ".marker").read_text() == "root\n" + assert (wt / "apps" / "server" / "node_modules" / "realdep" / "index.js").exists() + + # The relative workspace symlink was preserved AND resolves inside the worktree. + link = wt / "apps" / "server" / "node_modules" / "@scope" / "shared" + assert link.is_symlink() + resolved = os.path.realpath(link) + assert resolved.startswith(os.path.realpath(wt)) + assert not resolved.startswith(os.path.realpath(base)) + assert resolved == os.path.realpath(wt / "packages" / "shared") + + +@pytest.mark.skipif( + not clone_supported(tempfile.gettempdir()), + reason="filesystem does not support CoW/hardlink cloning", +) +def test_prime_worktree_by_clone_gates_on_sanity_probe(tmp_path): + """_prime_worktree_by_clone returns True only when the clone passes the probe, + and False (→ install fallback) when the probe fails or there is no probe.""" + import misterdev.agent as agent_mod + from unittest.mock import MagicMock + + base = tmp_path / "base" + base.mkdir() + _make_base(base) + wt = tmp_path / "wt" + wt.mkdir() + + project = MagicMock() + project.path = base + orch = agent_mod.ProjectOrchestrator() + + class _Tool: + def __init__(self, ok): + self.ok = ok + + def execute(self, project, command, cwd=None, timeout=None): + return self.ok, "" if self.ok else "error TS2307: cannot find module" + + # Probe passes -> primed by clone. + assert orch._prime_worktree_by_clone( + project, MagicMock(id="T"), wt, "tsc --version", 60, _Tool(True) + ) + # Probe fails -> decline, fall back to install. + assert not orch._prime_worktree_by_clone( + project, MagicMock(id="T"), tmp_path / "wt2", "tsc --version", 60, _Tool(False) + ) + # No probe command -> cannot verify, decline. + assert not orch._prime_worktree_by_clone( + project, MagicMock(id="T"), tmp_path / "wt3", None, 60, _Tool(True) + ) diff --git a/tests/test_doc_tool_default_mounted.py b/tests/test_doc_tool_default_mounted.py new file mode 100644 index 0000000..b4d4bcd --- /dev/null +++ b/tests/test_doc_tool_default_mounted.py @@ -0,0 +1,56 @@ +"""T2.4 — the documentation tool (context7 etc.) is mounted by default. + +A default build (no mcp config) should give the model a version-pinned library-docs +server so it can look things up instead of hallucinating APIs. Only the docs-category +core servers are mounted by default (not the whole core stack); opt out with +mcp.docs_tool: false, override with mcp.curated. +""" + +from pathlib import Path + +from misterdev.core.execution.project import Project +from misterdev.core.integration.mcp_registry import select_curated + + +class _Duck: + def __init__(self, config): + self.config = config + + def _host_exec_isolated(self): + return False + + def _mcp_cache_path(self): + return Path("/tmp/mcp-none.json") + + +_Duck._build_mcp = Project._build_mcp + + +def _mounted(config): + mgr = _Duck(config)._build_mcp() + return {s["name"] for s in mgr.servers} if mgr else set() + + +def test_select_curated_category_filter(): + docs = {s["name"] for s in select_curated(("core",), categories={"docs"})} + assert "context7" in docs + assert "git" not in docs # git is vcs, not docs + + +def test_docs_tool_mounted_by_default(): + names = _mounted({}) + docs = {s["name"] for s in select_curated(("core",), categories={"docs"})} + assert docs and docs <= names + # Only docs by default, not the whole curated core stack. + assert "git" not in names + + +def test_docs_tool_opt_out(): + assert _mounted( + {"mcp": {"docs_tool": False}} + ) == set() or "context7" not in _mounted({"mcp": {"docs_tool": False}}) + + +def test_explicit_curated_still_mounts_full_core(): + names = _mounted({"mcp": {"curated": True}}) + assert "git" in names and "context7" in names diff --git a/tests/test_doctor.py b/tests/test_doctor.py new file mode 100644 index 0000000..589c772 --- /dev/null +++ b/tests/test_doctor.py @@ -0,0 +1,144 @@ +"""misterdev doctor preflight: pure check routing, aggregation, and exit codes.""" + +import tempfile +from pathlib import Path +from unittest.mock import MagicMock + +from misterdev.core.execution import doctor as d + + +def test_clean_tree_routing(): + assert d.check_clean_tree("").status == d.PASS + assert d.check_clean_tree("3 files, e.g. env.ts").status == d.FAIL + + +def test_on_base_branch_routing(): + assert d.check_on_base_branch("main", "main").status == d.PASS + assert d.check_on_base_branch("feature", "main").status == d.WARN + assert d.check_on_base_branch("task/T-1-abc123", "main").status == d.FAIL + assert d.check_on_base_branch("doctor/abc123", "main").status == d.FAIL + assert d.check_on_base_branch(None, "main").status == d.FAIL + + +def test_models_routing(): + assert d.check_models(True, "good/model").status == d.PASS + assert d.check_models(False, "404 no endpoints").status == d.FAIL + + +def test_worktree_prime_routing(): + assert d.check_worktree_prime(None, None).status == d.PASS # nothing to prime + assert d.check_worktree_prime(True, True).status == d.PASS + assert d.check_worktree_prime(True, False).status == d.WARN # toolchain unresolved + assert d.check_worktree_prime(False, None).status == d.WARN # prime failed + + +def test_git_and_branch_and_worktree_and_requirements_warnings(): + assert d.check_git_repo(True).status == d.PASS + assert d.check_git_repo(False).status == d.WARN + assert d.check_leftover_task_branches([]).status == d.PASS + assert d.check_leftover_task_branches(["task/a", "task/b"]).status == d.WARN + assert d.check_dangling_worktrees([]).status == d.PASS + assert d.check_dangling_worktrees(["/x/.orchestrator/worktrees/a"]).status == d.WARN + assert d.check_requirements([]).status == d.PASS + assert d.check_requirements(["API_KEY"]).status == d.WARN + + +def test_aggregate_all_pass_exits_zero(): + checks = [ + d.check_clean_tree(""), + d.check_models(True, "m"), + d.check_requirements([]), + ] + agg = d.aggregate(checks) + assert agg["exit_code"] == 0 + assert agg["ready"] is True + assert agg["passed"] == 3 and agg["warnings"] == 0 and agg["failures"] == 0 + + +def test_aggregate_warnings_only_still_exits_zero(): + """Warnings inform but never block an unattended run.""" + checks = [ + d.check_git_repo(False), + d.check_leftover_task_branches(["task/a"]), + d.check_requirements(["X"]), + ] + agg = d.aggregate(checks) + assert agg["warnings"] == 3 + assert agg["failures"] == 0 + assert agg["exit_code"] == 0 + assert agg["ready"] is True + + +def test_aggregate_any_hard_failure_exits_nonzero(): + checks = [ + d.check_clean_tree(""), # pass + d.check_leftover_task_branches(["task/a"]), # warn + d.check_on_base_branch(None, "main"), # fail + ] + agg = d.aggregate(checks) + assert agg["failures"] == 1 + assert agg["exit_code"] == 1 + assert agg["ready"] is False + + +# --- run_doctor integration on a real temp git repo ------------------------- +def _git(root, *args): + import subprocess + + subprocess.run(["git", *args], cwd=root, check=True, capture_output=True, text=True) + + +def _clean_repo(root: Path): + _git(root, "init", "-b", "main") + _git(root, "config", "user.email", "t@t.t") + _git(root, "config", "user.name", "t") + (root / "f.txt").write_text("x\n") + _git(root, "add", "-A") + _git(root, "commit", "-q", "-m", "init") + + +def _doctor_on(root: Path, health=(True, "good/model")): + import misterdev.agent as agent_mod + + orch = agent_mod.ProjectOrchestrator() + project = MagicMock() + project.path = root + project.config = {} # non-node -> worktree probe short-circuits to pass + project.llm_client.health_check.return_value = health + orch._get_or_register = lambda _p: project # type: ignore[assignment] + return orch.run_doctor(str(root)) + + +def test_run_doctor_ready_on_clean_repo(): + with tempfile.TemporaryDirectory() as td: + root = Path(td) + _clean_repo(root) + result = _doctor_on(root) + assert result["exit_code"] == 0 + assert result["ready"] is True + names = {c.name: c.status for c in result["checks"]} + assert names["clean working tree"] == "pass" + assert names["on base branch"] == "pass" + assert names["models resolve"] == "pass" + + +def test_run_doctor_fails_on_dirty_tree(): + with tempfile.TemporaryDirectory() as td: + root = Path(td) + _clean_repo(root) + (root / "f.txt").write_text("changed\n") # dirty + result = _doctor_on(root) + assert result["exit_code"] == 1 + assert result["ready"] is False + names = {c.name: c.status for c in result["checks"]} + assert names["clean working tree"] == "fail" + + +def test_run_doctor_fails_when_model_unresolvable(): + with tempfile.TemporaryDirectory() as td: + root = Path(td) + _clean_repo(root) + result = _doctor_on(root, health=(False, "404 no endpoints")) + assert result["exit_code"] == 1 + names = {c.name: c.status for c in result["checks"]} + assert names["models resolve"] == "fail" diff --git a/tests/test_early_abort.py b/tests/test_early_abort.py deleted file mode 100644 index 9f35fc2..0000000 --- a/tests/test_early_abort.py +++ /dev/null @@ -1,59 +0,0 @@ -"""Process-signal early-abort: stop a non-converging task before the cost cap.""" - -from misterdev.core.execution.early_abort import ( - AttemptSignal, - ConvergenceMonitor, - fingerprint, -) - - -def test_fingerprint_ignores_incidental_detail(): - a = fingerprint("File /a/b.py line 42: NameError 'foo' at 0xdeadbeef") - b = fingerprint("File /c/d.py line 99: NameError 'foo' at 0xcafef00d") - assert a == b # same error, different paths/lines/addresses -> same fingerprint - - -def test_stuck_fires_after_repeats(): - m = ConvergenceMonitor(stuck_repeats=3, no_progress_window=0) - for _ in range(2): - m.update(AttemptSignal(5, "TypeError: bad arg at line 3")) - assert not m.should_abort()[0] # not yet 3 in a row - m.update(AttemptSignal(5, "TypeError: bad arg at line 7")) - abort, reason = m.should_abort() - assert abort and "stuck" in reason - - -def test_stuck_resets_when_error_changes(): - m = ConvergenceMonitor(stuck_repeats=3, no_progress_window=0) - m.update(AttemptSignal(1, "error A")) - m.update(AttemptSignal(1, "error A")) - m.update(AttemptSignal(1, "error B")) # different -> streak broken - assert not m.should_abort()[0] - - -def test_no_progress_fires_when_count_not_shrinking(): - m = ConvergenceMonitor(stuck_repeats=0, no_progress_window=3) - for c in (4, 4, 5): # never strictly decreases - m.update(AttemptSignal(c, "some error")) - abort, reason = m.should_abort() - assert abort and "no progress" in reason - - -def test_no_progress_does_not_fire_when_shrinking(): - m = ConvergenceMonitor(stuck_repeats=0, no_progress_window=3) - for c in (6, 4, 2): # strictly decreasing -> making progress - m.update(AttemptSignal(c, "e")) - assert not m.should_abort()[0] - - -def test_needs_evidence_before_aborting(): - m = ConvergenceMonitor(stuck_repeats=3, no_progress_window=3) - m.update(AttemptSignal(9, "boom")) - assert not m.should_abort()[0] # a single attempt never aborts - - -def test_triggers_can_be_disabled(): - m = ConvergenceMonitor(stuck_repeats=0, no_progress_window=0) - for _ in range(10): - m.update(AttemptSignal(5, "same error")) - assert not m.should_abort()[0] # both triggers off -> never aborts diff --git a/tests/test_env_learnings.py b/tests/test_env_learnings.py new file mode 100644 index 0000000..63b593c --- /dev/null +++ b/tests/test_env_learnings.py @@ -0,0 +1,130 @@ +"""Per-project cross-run environment memory.""" + +import json +import tempfile +from pathlib import Path + +from misterdev.core.execution.env_learnings import EnvLearnings +from misterdev.utils.file_utils import orchestrator_state_file + + +def test_read_write_round_trip(): + with tempfile.TemporaryDirectory() as td: + e = EnvLearnings( + worktree_setup_command="pnpm install --prefer-offline", + worktree_healthcheck_command="npx --no-install tsc --version", + max_workers=3, + ordering_constraints=["dashboard build must precede server test"], + ) + e.save(td) + back = EnvLearnings.load(td) + assert back == e + # And the on-disk form is the documented tiny schema. + on_disk = json.loads( + orchestrator_state_file(td, "env_learnings.json").read_text() + ) + assert on_disk["version"] == 1 + assert on_disk["max_workers"] == 3 + assert on_disk["ordering_constraints"] == [ + "dashboard build must precede server test" + ] + + +def test_missing_file_loads_empty(): + with tempfile.TemporaryDirectory() as td: + e = EnvLearnings.load(td) + assert e == EnvLearnings() + assert e.ordering_constraints == [] + + +def test_corrupt_file_self_heals_to_empty(): + with tempfile.TemporaryDirectory() as td: + orchestrator_state_file(td, "env_learnings.json").write_text("{not json") + assert EnvLearnings.load(td) == EnvLearnings() + + +def test_apply_fills_unset_config(): + e = EnvLearnings( + worktree_setup_command="npm ci", + worktree_healthcheck_command="node -e 0", + max_workers=2, + ) + cfg = {} + applied = e.apply_to_config(cfg) + assert cfg["orchestrator"]["worktree_setup_command"] == "npm ci" + assert cfg["orchestrator"]["worktree_healthcheck_command"] == "node -e 0" + assert cfg["orchestrator"]["max_workers"] == 2 + assert len(applied) == 3 + + +def test_explicit_config_always_wins(): + """A learned value never overrides an explicit project.yaml value; it only + fills a key the user left unset.""" + e = EnvLearnings(worktree_setup_command="npm ci", max_workers=2) + cfg = {"orchestrator": {"max_workers": 8}} # user set max_workers explicitly + applied = e.apply_to_config(cfg) + assert cfg["orchestrator"]["max_workers"] == 8 # explicit wins + assert cfg["orchestrator"]["worktree_setup_command"] == "npm ci" # gap filled + assert applied == ["orchestrator.worktree_setup_command='npm ci'"] + + +def test_none_learnings_apply_nothing(): + cfg = {"orchestrator": {}} + assert EnvLearnings().apply_to_config(cfg) == [] + assert cfg == {"orchestrator": {}} # untouched + + +def test_record_persists_reduced_workers_and_clears_on_recovery(): + """_record_env_learnings persists a contention-driven worker reduction, and a + later run that held full concurrency clears it (no permanent pin).""" + from unittest.mock import MagicMock + import misterdev.agent as agent_mod + + with tempfile.TemporaryDirectory() as td: + orch = agent_mod.ProjectOrchestrator() + project = MagicMock() + project.path = Path(td) + project.config = {} + + # Run 1 backed off from 4 to 2 workers -> persist the reduction. + project.env_settled_workers = 2 + project.env_base_workers = 4 + orch._record_env_learnings(project) + assert EnvLearnings.load(td).max_workers == 2 + + # Run 2 held/recovered to the base -> clear the stale reduction. + project.env_settled_workers = 4 + project.env_base_workers = 4 + orch._record_env_learnings(project) + assert EnvLearnings.load(td).max_workers is None + + +def test_record_is_noop_without_settled_workers(): + """A run that never adapted (no settled value) leaves a prior learned + max_workers untouched.""" + from unittest.mock import MagicMock + import misterdev.agent as agent_mod + + with tempfile.TemporaryDirectory() as td: + EnvLearnings(max_workers=2).save(td) + orch = agent_mod.ProjectOrchestrator() + project = MagicMock() + project.path = Path(td) + project.config = {} + # No env_settled_workers/env_base_workers attributes present. + del project.env_settled_workers + del project.env_base_workers + orch._record_env_learnings(project) + assert EnvLearnings.load(td).max_workers == 2 # preserved + + +def test_ordering_constraints_persist_but_do_not_tune_config(): + """Ordering constraints round-trip in the ledger but are advisory — they are + not written into the runtime config.""" + with tempfile.TemporaryDirectory() as td: + EnvLearnings(ordering_constraints=["a before b"]).save(td) + loaded = EnvLearnings.load(td) + assert loaded.ordering_constraints == ["a before b"] + cfg = {} + loaded.apply_to_config(cfg) + assert cfg == {} # no config knob for ordering constraints diff --git a/tests/test_error_classifier.py b/tests/test_error_classifier.py index 125914b..e40b2dc 100644 --- a/tests/test_error_classifier.py +++ b/tests/test_error_classifier.py @@ -219,3 +219,16 @@ def test_link_error_guidance_is_language_neutral(): def test_every_indicator_category_has_guidance(): for category in _TIE_BREAK_ORDER + [ErrorCategory.UNKNOWN]: assert FIX_GUIDANCE.get(category, "").strip() + + +def test_csharp_code_requires_roslyn_error_prefix(): + # A genuine Roslyn diagnostic still classifies by its code. + real = "Program.cs(5,10): error CS0103: The name 'x' does not exist" + assert classify_error(real) == ErrorCategory.MISSING_SYMBOL + + +def test_bare_csharp_code_substring_does_not_override_real_category(): + # A CSxxxx: substring appearing in prose/a doc URL/an assertion log must NOT + # hijack classification via the fast path (it is not a Roslyn error line). + log = "thread panicked at assertion failed: left == right. See docs at CS0103: notes" + assert classify_error(log) != ErrorCategory.MISSING_SYMBOL diff --git a/tests/test_escalation.py b/tests/test_escalation.py new file mode 100644 index 0000000..d9b9968 --- /dev/null +++ b/tests/test_escalation.py @@ -0,0 +1,86 @@ +"""Escalation ladder: rung selection and the infra-never-advances guarantee.""" + +from misterdev.core.execution.escalation import RUNGS, choose_rung, should_count_failure + + +def test_ladder_climbs_with_code_failures(): + assert choose_rung(0) == "normal" + assert choose_rung(1) == "widen_context" + assert choose_rung(2) == "full_rewrite" + assert choose_rung(3) == "decompose" + assert choose_rung(9) == "decompose" # decompose is terminal + + +def test_ladder_has_a_structural_full_rewrite_rung(): + # T3.1: a real, structurally-different rung sits between the context-widening + # reprompt and the model swap. + assert "full_rewrite" in RUNGS + assert ( + RUNGS.index("widen_context") + < RUNGS.index("full_rewrite") + < RUNGS.index("stronger_model") + ) + assert choose_rung(2) == "full_rewrite" + + +def test_custom_thresholds(): + kw = dict(widen_after=2, rewrite_after=3, model_after=4, decompose_after=6) + assert choose_rung(1, **kw) == "normal" + assert choose_rung(2, **kw) == "widen_context" + assert choose_rung(3, **kw) == "full_rewrite" + assert choose_rung(4, **kw) == "stronger_model" + assert choose_rung(6, **kw) == "decompose" + + +def test_misordered_thresholds_stay_monotonic(): + # Highest threshold met wins, so a nonsensical config still yields a defined + # (never lower-than-expected) rung. + assert ( + choose_rung(5, widen_after=3, model_after=1, decompose_after=2) == "decompose" + ) + + +def test_should_count_failure_ignores_infra(): + for infra in ( + "Command timed out after 120s", + "waiting for the lock on the store", + "ENOSPC: no space left on device", + "JavaScript heap out of memory", + ): + assert should_count_failure(infra) is False + + +def test_should_count_failure_counts_real_code_errors(): + for code in ( + "error TS2345: Argument of type 'string' is not assignable", + "AssertionError: expected 3 got 4", + "SyntaxError: Unexpected token )", + ): + assert should_count_failure(code) is True + + +def test_infra_failures_never_advance_the_ladder(): + """A counter that only increments on should_count_failure escalates on CODE + failures and holds across interleaved INFRA failures.""" + outputs = [ + "Command timed out after 120s", # infra -> hold at normal + "error TS2345: bad type", # code -> widen + "ELOCK: store is locked", # infra -> hold at widen + "AssertionError: 1 != 2", # code -> full_rewrite + "operation timed out", # infra -> hold at full_rewrite + "error TS2304: name not found", # code -> decompose + ] + code_failures = 0 + rungs = [] + for out in outputs: + if should_count_failure(out): + code_failures += 1 + rungs.append(choose_rung(code_failures)) + assert rungs == [ + "normal", + "widen_context", + "widen_context", + "full_rewrite", + "full_rewrite", + "decompose", + ] diff --git a/tests/test_escalation_substeps_execute.py b/tests/test_escalation_substeps_execute.py new file mode 100644 index 0000000..cfa0bfc --- /dev/null +++ b/tests/test_escalation_substeps_execute.py @@ -0,0 +1,64 @@ +"""T4.1 — decompose-rung sub-steps are EXECUTED as real child tasks. + +`_escalate_decompose` used to only record sub-steps and park the task. It now runs +each sub-step through the real per-task gate via `self.execute`, one recursion level +deep (a sub-step can never itself re-decompose). If all sub-steps complete, the parent +is satisfied; at depth >= 1 it falls back to deferring. +""" + +from types import SimpleNamespace +from unittest.mock import MagicMock + +from misterdev.core.models import ExecutionResult, Task +from misterdev.task_executors.markdown_plan_executor import MarkdownPlanExecutor + + +class _Spy(MarkdownPlanExecutor): + def __init__(self, child_status="completed"): + self.execute_calls = [] + self._child_status = child_status + + def execute(self, task, project, use_git_branch=True, _depth=0): + self.execute_calls.append((task.id, _depth)) + return ExecutionResult(status=self._child_status, message="") + + def _record_invented_tools(self, project, task, resolved): + pass + + +def _parent(): + return Task( + id="BIG", + description="big task", + project_ref=".", + acceptance_criteria="Do part one thoroughly\nDo part two thoroughly", + files_to_modify=["a.py"], + processor_data={}, + ) + + +def _project(): + return SimpleNamespace(task_manager=MagicMock(), topography=None) + + +def test_substeps_execute_as_real_children(): + ex = _Spy() + result = ex._escalate_decompose(_project(), _parent(), "") + # Two acceptance lines -> two sub-steps, each executed as a child at depth+1. + assert len(ex.execute_calls) == 2 + assert all(d == 1 for _, d in ex.execute_calls) + assert result.status == "completed" + + +def test_failed_substep_defers_parent(): + ex = _Spy(child_status="failed") + result = ex._escalate_decompose(_project(), _parent(), "") + assert len(ex.execute_calls) == 2 + assert result.status == "deferred" + + +def test_no_recursion_at_depth_one(): + ex = _Spy() + result = ex._escalate_decompose(_project(), _parent(), "", _depth=1) + assert ex.execute_calls == [] # deferred, not executed + assert result.status == "deferred" diff --git a/tests/test_evolution_pass_benchmark_gated.py b/tests/test_evolution_pass_benchmark_gated.py new file mode 100644 index 0000000..63f8d09 --- /dev/null +++ b/tests/test_evolution_pass_benchmark_gated.py @@ -0,0 +1,64 @@ +"""T5.2 — a scheduled evolution pass runs live behind the benchmark, lock-guarded. + +`run_scheduled_evolution` is the schedulable entrypoint: it runs ONE live evolution +pass (which only promotes a self-edit that passes the benchmark) under an exclusive +lock, so overlapping scheduled runs cannot corrupt the shared archive. A busy lock is +a clean no-op. The benchmark-gating itself lives in run_evolution (live path) and is +injected here as a fake. +""" + +from types import SimpleNamespace + +from misterdev.core.evolution.scheduled import run_scheduled_evolution + + +def test_runs_one_live_benchmark_gated_pass(tmp_path): + calls = {} + + def fake_run(project, bdir, wdir, *, gate_commands, live=False, **kw): + calls["live"] = live + calls["gate"] = gate_commands + return "RESULT" + + out = run_scheduled_evolution( + SimpleNamespace(path=tmp_path), + "bench", + "work", + gate_commands={"test_command": "pytest"}, + _run=fake_run, + ) + assert out == "RESULT" + assert calls["live"] is True # scheduled pass is ALWAYS live+benchmark-gated + assert calls["gate"] == {"test_command": "pytest"} + # The lock is released after the pass completes. + assert not (tmp_path / ".orchestrator" / "evolution" / "scheduled.lock").exists() + + +def test_busy_lock_is_skipped(tmp_path): + lock = tmp_path / ".orchestrator" / "evolution" / "scheduled.lock" + lock.parent.mkdir(parents=True, exist_ok=True) + lock.write_text("held") + called = {"n": 0} + + def fake_run(*a, **k): + called["n"] += 1 + return "X" + + out = run_scheduled_evolution( + SimpleNamespace(path=tmp_path), "b", "w", gate_commands={}, _run=fake_run + ) + assert out is None # overlap is skipped, not queued + assert called["n"] == 0 + + +def test_lock_released_even_when_pass_raises(tmp_path): + def boom(*a, **k): + raise RuntimeError("pass failed") + + try: + run_scheduled_evolution( + SimpleNamespace(path=tmp_path), "b", "w", gate_commands={}, _run=boom + ) + except RuntimeError: + pass + assert not (tmp_path / ".orchestrator" / "evolution" / "scheduled.lock").exists() diff --git a/tests/test_failurelog_runtime_readback.py b/tests/test_failurelog_runtime_readback.py new file mode 100644 index 0000000..83bcbb0 --- /dev/null +++ b/tests/test_failurelog_runtime_readback.py @@ -0,0 +1,70 @@ +"""T5.1 — the FailureLog is read back at runtime to inform later attempts. + +`_failure_priors` loads THIS project's FailureLog and surfaces the prior failures of +the current task (by id) into the edit context, so a repeated attempt within the same +run does not rediscover an error it already hit. Best-effort: no log / no match -> "". +""" + +import json +from types import SimpleNamespace + +from misterdev.task_executors.markdown_plan_executor.context_mixin import ContextMixin + + +class _Ex(ContextMixin): + pass + + +def _project(tmp_path): + return SimpleNamespace(path=tmp_path) + + +def _write_failures(tmp_path, rows): + d = tmp_path / ".orchestrator" + d.mkdir(parents=True, exist_ok=True) + (d / "failures.jsonl").write_text( + "\n".join(json.dumps(r) for r in rows) + "\n", encoding="utf-8" + ) + + +def test_prior_failures_for_task_are_surfaced(tmp_path): + _write_failures( + tmp_path, + [ + { + "name": "T1", + "language": "python", + "error": "AssertionError: 1 != 2", + "category": "test_assertion", + }, + { + "name": "T1", + "language": "python", + "error": "TypeError: bad arg", + "category": "wrong_type", + }, + { + "name": "OTHER", + "language": "python", + "error": "unrelated", + "category": "x", + }, + ], + ) + out = _Ex()._failure_priors(_project(tmp_path), SimpleNamespace(id="T1")) + assert "Prior failures" in out + assert "AssertionError: 1 != 2" in out + assert "TypeError: bad arg" in out + assert "unrelated" not in out # another task's failure is not surfaced + + +def test_no_log_returns_empty(tmp_path): + assert _Ex()._failure_priors(_project(tmp_path), SimpleNamespace(id="T1")) == "" + + +def test_no_match_returns_empty(tmp_path): + _write_failures( + tmp_path, + [{"name": "OTHER", "language": "python", "error": "e", "category": "c"}], + ) + assert _Ex()._failure_priors(_project(tmp_path), SimpleNamespace(id="T1")) == "" diff --git a/tests/test_gatekeeper_diff_cache.py b/tests/test_gatekeeper_diff_cache.py new file mode 100644 index 0000000..0ce74c0 --- /dev/null +++ b/tests/test_gatekeeper_diff_cache.py @@ -0,0 +1,56 @@ +"""H3 — the Gatekeeper git-diff is computed once per run_gates, not once per gate. + +G5/G6/G9 each called `_iter_diff_added_lines` (3+ git subprocesses + untracked-file +reads), but the tree is identical across gates within one run_gates. Memoize it and +invalidate at the start of each run_gates so a later run (a changed tree) recomputes. +""" + +import misterdev.core.verification.gatekeeper as gk_mod +from misterdev.core.verification.gatekeeper import GateKeeper + + +class _R: + def __init__(self, stdout, returncode=0): + self.stdout = stdout + self.returncode = returncode + + +def _gk(tmp_path): + gk = GateKeeper.__new__(GateKeeper) + gk.project_path = tmp_path + return gk + + +def test_diff_is_memoized_within_a_run(tmp_path, monkeypatch): + calls = {"n": 0} + + def fake_run_git(cmd, path): + calls["n"] += 1 + return _R("true" if "rev-parse" in cmd else "") + + monkeypatch.setattr(gk_mod, "run_git", fake_run_git) + gk = _gk(tmp_path) + + gk._iter_diff_added_lines() + after_first = calls["n"] + assert after_first > 0 + gk._iter_diff_added_lines() + assert calls["n"] == after_first # 2nd call served from cache; no new subprocesses + + +def test_invalidation_forces_recompute(tmp_path, monkeypatch): + calls = {"n": 0} + monkeypatch.setattr( + gk_mod, + "run_git", + lambda cmd, path: ( + calls.__setitem__("n", calls["n"] + 1), + _R("true" if "rev-parse" in cmd else ""), + )[1], + ) + gk = _gk(tmp_path) + gk._iter_diff_added_lines() + after_first = calls["n"] + gk._diff_cache_valid = False # what run_gates does at its start + gk._iter_diff_added_lines() + assert calls["n"] > after_first # a fresh run recomputes diff --git a/tests/test_goal_check_wiring.py b/tests/test_goal_check_wiring.py index 961ad71..e95cf5d 100644 --- a/tests/test_goal_check_wiring.py +++ b/tests/test_goal_check_wiring.py @@ -150,7 +150,9 @@ def test_phase4_flags_default_off_leave_loop_unchanged(): assert orch["goal_check"] is False assert orch["block_on_goal_gap"] is False assert orch["mutation_gate"] is False - assert orch["spec_as_tests"] is False + # spec_as_tests is default-on (T1.1), but advisory (spec_as_tests_block off). + assert orch["spec_as_tests"] is True + assert orch["spec_as_tests_block"] is False # And the gate guard reads the same default, so the goal check is skipped. assert get_setting({}, "orchestrator", "goal_check") is False diff --git a/tests/test_held_out_oracle.py b/tests/test_held_out_oracle.py deleted file mode 100644 index fdb0ce5..0000000 --- a/tests/test_held_out_oracle.py +++ /dev/null @@ -1,73 +0,0 @@ -"""Held-out oracle: reject an acceptance test that passes against a stub.""" - -from misterdev.core.execution.outcomes import GREEN, RED, SKIP -from misterdev.core.verification.held_out_oracle import ( - check_oracle, - looks_trivial, - stub_python, -) - -_OLD = "def classify(score):\n return 'x'\n" -_NEW = "def classify(score):\n if score >= 60:\n return 'pass'\n return 'fail'\n" - - -def test_stub_python_blanks_changed_function(): - stub = stub_python(_NEW, {1, 2, 3}) - assert "raise NotImplementedError" in stub - assert "return 'pass'" not in stub - - -def test_stub_python_skips_untouched_function(): - src = "def a():\n return 1\ndef b():\n return 2\n" - stub = stub_python(src, {1}) # only a() changed - assert stub.count("raise NotImplementedError") == 1 - assert "return 2" in stub # b() untouched - - -def test_stub_python_none_when_no_changed_function(): - assert stub_python(_NEW, set()) is None - - -def test_real_oracle_passes_when_stub_fails(tmp_path): - (tmp_path / "m.py").write_text(_NEW) - - def runner(cmd, timeout): - # A REAL test: fails on the stub (NotImplementedError), passes on the fix. - return "raise NotImplementedError" not in (tmp_path / "m.py").read_text(), "" - - res = check_oracle(tmp_path, "m.py", _OLD, _NEW, "pytest", runner=runner) - assert res.status == GREEN - assert (tmp_path / "m.py").read_text() == _NEW # restored - - -def test_trivial_oracle_flagged_when_stub_passes(tmp_path): - (tmp_path / "m.py").write_text(_NEW) - - def runner(cmd, timeout): - return True, "" # passes no matter what -> trivial oracle - - res = check_oracle(tmp_path, "m.py", _OLD, _NEW, "pytest", runner=runner) - assert res.status == RED - assert "trivial oracle" in res.reason - assert (tmp_path / "m.py").read_text() == _NEW # restored even on RED - - -def test_skips_non_python_and_no_command(tmp_path): - (tmp_path / "m.rs").write_text("fn f() {}") - assert ( - check_oracle( - tmp_path, "m.rs", "a", "b", "cargo test", lambda c, t: (True, "") - ).status - == SKIP - ) - assert ( - check_oracle(tmp_path, "m.py", _OLD, _NEW, "", lambda c, t: (True, "")).status - == SKIP - ) - - -def test_looks_trivial_smell(): - assert looks_trivial("def t():\n assert isinstance(r, str)\n") - assert looks_trivial("def t():\n assert r is not None\n") - assert looks_trivial("def t():\n pass\n") # no assertion at all - assert not looks_trivial("def t():\n assert classify(60) == 'pass'\n") diff --git a/tests/test_independent_judge.py b/tests/test_independent_judge.py new file mode 100644 index 0000000..b0b21c7 --- /dev/null +++ b/tests/test_independent_judge.py @@ -0,0 +1,74 @@ +"""T1.2 — the judge/critic must run on a model INDEPENDENT of the generator. + +`build_independent_call` already warns when a model is set but the client can't +switch, and when no model is configured. The silent hole: when the configured judge +model EQUALS the generator's current model, it claims independence and routes to the +same model — zero independence, no signal. This asserts that same-model is detected +(warned) and NOT treated as independent routing, while a genuinely different model +still routes. +""" + +import contextlib + +from misterdev.core.verification.independent import build_independent_call + + +class _Switchable: + def __init__(self, model="gen-model"): + self._model = model + self.entered = [] + + @property + def model(self): + return self._model + + def generate_code(self, prompt, system=""): + return "verdict" + + @contextlib.contextmanager + def with_model(self, m): + self.entered.append(m) + prev, self._model = self._model, m + try: + yield + finally: + self._model = prev + + +class _NoSwitch: + model = "gen-model" + + def generate_code(self, prompt, system=""): + return "verdict" + + +def test_same_model_is_not_routed_as_independent(caplog): + client = _Switchable("gen-model") + call = build_independent_call(client, "", "gen-model", "Judge") + assert call is not None + call("prompt") + # Same model as the generator -> not real independence -> must NOT route through + # with_model (which would falsely imply an independent check). + assert client.entered == [] + assert any("same" in r.message.lower() for r in caplog.records) + + +def test_distinct_model_routes_independently(): + client = _Switchable("gen-model") + call = build_independent_call(client, "", "other-model", "Judge") + call("prompt") + assert client.entered == ["other-model"] + + +def test_no_model_runs_on_generator_without_routing(): + client = _Switchable("gen-model") + call = build_independent_call(client, "", None, "Judge") + call("prompt") + assert client.entered == [] + + +def test_unroutable_model_still_warns_and_runs(): + client = _NoSwitch() + call = build_independent_call(client, "", "other-model", "Judge") + assert call is not None + assert call("prompt") == "verdict" diff --git a/tests/test_infra.py b/tests/test_infra.py new file mode 100644 index 0000000..598fdda --- /dev/null +++ b/tests/test_infra.py @@ -0,0 +1,134 @@ +"""Infra-vs-code failure classification and the self-healing test rerun.""" + +from pathlib import Path +from types import SimpleNamespace + +from misterdev.agent_helpers import worktree_setup_command +from misterdev.core.execution.infra import infra_failure +from misterdev.task_executors.markdown_plan_executor import MarkdownPlanExecutor + + +def test_infra_failure_flags_environment_faults(): + for out in ( + "Command timed out after 120s: pnpm --filter @countless/server typecheck", + "Error: Cannot find module 'hono'", + "sh: tsc: command not found", + "This is not the tsc command you are looking for", + "ENOSPC: no space left on device", + "FATAL ERROR: JavaScript heap out of memory", + "waiting for the lock on the store", + "EMFILE: too many open files", + ): + assert infra_failure(out), out + + +def test_infra_failure_ignores_real_code_errors(): + for out in ( + "AssertionError: expected 3 got 4", + "error TS2345: Argument of type 'string' is not assignable to 'number'", + "Test Files 1 failed | 20 passed", + "SyntaxError: Unexpected token )", + "2 failed, 5 passed in 0.3s", + ): + assert infra_failure(out) is None, out + + +def test_confirm_flaky_self_heals_on_infra_even_with_reruns_zero(): + """A test gate that fails on an environment fault (a timeout) is re-run once + even when flaky_reruns is 0 — and a clean re-run is treated as a flake, not a + reverted code failure. A real code error is NOT re-run when reruns is 0.""" + e = MarkdownPlanExecutor() + project = SimpleNamespace() + calls = {"n": 0} + + def fake_run(_project, _cmd, timeout=None, cwd=None): + calls["n"] += 1 + return (True, "ok") # the re-run passes cleanly + + e._run_command = fake_run # type: ignore[assignment] + + # Infra signature + reruns=0 → self-heals (re-runs, non-reproducing → flake). + assert e._confirm_flaky( + project, "pnpm test", "Command timed out after 180s", 5, None, 0 + ) + assert calls["n"] >= 1 + + # A real code failure + reruns=0 → no re-run, treated as a genuine failure. + calls["n"] = 0 + assert not e._confirm_flaky( + project, "pnpm test", "AssertionError: 3 != 4", 5, None, 0 + ) + assert calls["n"] == 0 + + +def _gate_executor(outputs, reprimes): + """A MarkdownPlanExecutor whose gate runs return queued (ok, output) pairs and + whose worktree re-prime is recorded, not executed.""" + e = MarkdownPlanExecutor() + queue = list(outputs) + + def fake_run(_project, _cmd, timeout=None, cwd=None): + return queue.pop(0) + + e._run_command = fake_run # type: ignore[assignment] + e._reprime_worktree_deps = lambda _project: reprimes.append(True) # type: ignore[assignment] + return e + + +def test_run_gate_reprimes_and_reruns_once_on_infra(): + """An infra-signatured gate failure re-primes the worktree and re-runs the + gate exactly once; a green re-run makes the gate pass.""" + reprimes: list = [] + e = _gate_executor( + [(False, "Command timed out after 120s: tsc"), (True, "ok")], reprimes + ) + ok, out = e._run_gate(SimpleNamespace(), "npm run typecheck", 120, None) + assert ok and out == "ok" + assert reprimes == [True] # re-primed exactly once + + +def test_run_gate_does_not_rerun_on_code_error(): + """A plain code failure (a type error) carries no infra signature, so the gate + is returned as-is with no re-prime and no re-run.""" + reprimes: list = [] + e = _gate_executor( + [(False, "error TS2345: Argument of type 'string' is not assignable")], + reprimes, + ) + ok, out = e._run_gate(SimpleNamespace(), "npm run typecheck", 120, None) + assert not ok and "TS2345" in out + assert reprimes == [] # never re-primed on a code failure + + +def test_run_gate_passes_through_green_without_reprime(): + """A gate that passes on the first run returns immediately, no re-prime.""" + reprimes: list = [] + e = _gate_executor([(True, "ok")], reprimes) + ok, out = e._run_gate(SimpleNamespace(), "cargo build", 120, None) + assert ok and out == "ok" + assert reprimes == [] + + +def test_run_gate_infra_rerun_still_red_returns_rerun_output(): + """When the re-primed re-run still fails, the gate returns the re-run's output + (not the first), so the retry context reflects the latest state.""" + reprimes: list = [] + e = _gate_executor( + [(False, "Cannot find module 'hono'"), (False, "error TS2304: name X")], + reprimes, + ) + ok, out = e._run_gate(SimpleNamespace(), "npm run typecheck", 120, None) + assert not ok and "TS2304" in out + assert reprimes == [True] + + +def test_worktree_setup_command_resolution(tmp_path: Path): + """Explicit config wins and "" disables; otherwise the command is auto-detected + from the project's lockfile, with a bare package.json falling back to install.""" + assert worktree_setup_command({}, tmp_path) is None + cfg = {"orchestrator": {"worktree_setup_command": "make deps"}} + assert worktree_setup_command(cfg, tmp_path) == "make deps" + off = {"orchestrator": {"worktree_setup_command": ""}} + assert worktree_setup_command(off, tmp_path) is None + (tmp_path / "package-lock.json").write_text("{}", encoding="utf-8") + assert worktree_setup_command({}, tmp_path) == "npm ci" diff --git a/tests/test_integration_gate_count.py b/tests/test_integration_gate_count.py index e9a5657..be5b422 100644 --- a/tests/test_integration_gate_count.py +++ b/tests/test_integration_gate_count.py @@ -389,3 +389,58 @@ class _Proj: results = orch._validate_targets(_Proj(), None) assert results[0]["ok"] is True + + +# --- C1: build-break waves must be reverted (not waved through as unparseable) --- + + +class _BrokenExec: + """A wave that BREAKS the build: the suite command fails with a compile/ + collection error (no parseable test count), on every run.""" + + def __init__(self): + self.reverted = [] + + def _run_command(self, project, cmd, timeout=0, cwd=None): + return False, ( + "ERROR collecting tests/test_x.py\n" + "ModuleNotFoundError: No module named 'brokenpkg'\n" + ) + + def find_task_commit(self, project, tid): + return f"sha-{tid}" + + def revert_task_commit(self, project, sha): + self.reverted.append(sha) + return True + + +def test_count_gate_reverts_when_wave_breaks_the_build(): + # A build/collection break is strictly worse than a counted red baseline, even + # though it yields no parseable count. It must be reverted, not waved through. + orch = ProjectOrchestrator() + ex = _BrokenExec() + reverted = orch._integration_gate_count( + None, ex, "t", [_task("T-1")], 1, baseline_failures=2 + ) + assert reverted == ["T-1"] + assert ex.reverted == ["sha-T-1"] + + +def test_identity_gate_reverts_when_wave_breaks_the_build(): + orch = ProjectOrchestrator() + baseline = _ids(["test_alpha"]) + ex = _BrokenExec() + reverted = orch._integration_gate_ids( + _Proj(), ex, "t", [_task("T-1")], 1, baseline + ) + assert reverted == ["T-1"] + + +def test_count_gate_ambiguous_unparseable_still_not_reverted(): + # Control: a failing-but-unparseable run that is NOT a build break stays + # ambiguous — the deliberate don't-revert-on-a-number-we-cant-read behavior. + orch = ProjectOrchestrator() + ex = _FakeExec([9], unparseable=True) + assert orch._integration_gate_count(None, ex, "t", [_task("T-1")], 1, 2) == [] + assert ex.reverted == [] diff --git a/tests/test_jobs.py b/tests/test_jobs.py new file mode 100644 index 0000000..f47e02e --- /dev/null +++ b/tests/test_jobs.py @@ -0,0 +1,145 @@ +"""Background job registry and cooperative-stop plumbing.""" + +import threading +import time + +import pytest + +from misterdev.agent import ProjectOrchestrator +from misterdev.core.execution.jobs import JobRegistry + + +def _wait(reg: JobRegistry, run_id: str, want: str, timeout: float = 3.0) -> None: + deadline = time.monotonic() + timeout + while time.monotonic() < deadline: + state = reg.status(run_id) + if state and state["status"] == want: + return + time.sleep(0.01) + raise AssertionError(f"job {run_id} never reached {want!r}: {reg.status(run_id)}") + + +def test_start_runs_target_and_captures_result(): + reg = JobRegistry() + run_id = reg.start("build", "/p", lambda: "OK-REPORT") + _wait(reg, run_id, "succeeded") + state = reg.status(run_id) + assert state["kind"] == "build" + assert state["result"] == "OK-REPORT" + assert state["error"] is None + assert state["ended_at"] is not None + + +def test_exception_marks_job_failed_not_lost(): + reg = JobRegistry() + + def boom() -> str: + raise ValueError("boom") + + run_id = reg.start("build", "/p", boom) + _wait(reg, run_id, "failed") + assert reg.status(run_id)["error"] == "boom" + + +def test_one_job_per_project_is_refused(): + reg = JobRegistry() + hold = threading.Event() + run_id = reg.start("build", "/p", lambda: hold.wait(3) or "x") + assert reg.status(run_id)["status"] == "running" + with pytest.raises(RuntimeError): + reg.start("run", "/p", lambda: "y") + # A different project is allowed to run concurrently. + other = reg.start("run", "/q", lambda: "z") + _wait(reg, other, "succeeded") + hold.set() + _wait(reg, run_id, "succeeded") + + +def test_stop_signals_hook_and_marks_stopped(): + reg = JobRegistry() + release = threading.Event() + stopped = {"seen": False} + + def target() -> str: + release.wait(3) + return "partial" if stopped["seen"] else "full" + + def stop_hook() -> None: + stopped["seen"] = True + release.set() + + run_id = reg.start("build", "/p", target, stop_hook=stop_hook) + assert reg.status(run_id)["status"] == "running" + assert reg.stop(run_id) is True + _wait(reg, run_id, "stopped") + assert stopped["seen"] is True + # A stop-labelled result, so the reader isn't misled by the budget mechanism. + assert reg.status(run_id)["result"].startswith("Stopped by request") + + +def test_stop_unknown_or_finished_returns_false(): + reg = JobRegistry() + assert reg.stop("nope") is False + run_id = reg.start("build", "/p", lambda: "done") + _wait(reg, run_id, "succeeded") + assert reg.stop(run_id) is False + + +def test_list_jobs_reports_all(): + reg = JobRegistry() + a = reg.start("build", "/a", lambda: "1") + b = reg.start("run", "/b", lambda: "2") + _wait(reg, a, "succeeded") + _wait(reg, b, "succeeded") + ids = {j["run_id"] for j in reg.list_jobs()} + assert {a, b} <= ids + + +def test_status_unknown_run_id_is_none(): + assert JobRegistry().status("missing") is None + + +def test_finished_jobs_are_evicted_beyond_cap(): + reg = JobRegistry(max_finished=3) + ids = [] + for i in range(6): + run_id = reg.start("build", f"/p{i}", lambda: "done") + _wait(reg, run_id, "succeeded") + ids.append(run_id) + # Only the most recent 3 finished jobs are retained; the oldest are gone. + kept = {j["run_id"] for j in reg.list_jobs()} + assert len(kept) == 3 + assert kept == set(ids[-3:]) + assert reg.status(ids[0]) is None + + +def test_running_jobs_are_never_evicted(): + reg = JobRegistry(max_finished=1) + hold = threading.Event() + long_running = reg.start("build", "/long", lambda: hold.wait(3) or "x") + # Churn several finished jobs past the cap while the long one runs. + for i in range(4): + done = reg.start("build", f"/p{i}", lambda: "done") + _wait(reg, done, "succeeded") + assert reg.status(long_running)["status"] == "running" + hold.set() + _wait(reg, long_running, "succeeded") + + +def test_request_stop_trips_active_client_budget(): + orch = ProjectOrchestrator() + + class _Client: + _budget = 5.0 + + client = _Client() + orch._active_client = client + orch.request_stop() + assert orch._stop_requested is True + assert client._budget == 0.0 + + +def test_request_stop_without_client_still_sets_flag(): + orch = ProjectOrchestrator() + orch.request_stop() + assert orch._stop_requested is True diff --git a/tests/test_jobs_persistence.py b/tests/test_jobs_persistence.py new file mode 100644 index 0000000..2435232 --- /dev/null +++ b/tests/test_jobs_persistence.py @@ -0,0 +1,82 @@ +"""A1 — async jobs persist across an MCP server restart. + +A JobRegistry with a store_path writes job state to disk, so a client can still poll +a run_id after the server restarts. A job persisted as "running" (its process died +mid-run) loads back as "interrupted", not as if it were still progressing. +""" + +import json + +from misterdev.core.execution.jobs import JobRegistry + + +def _wait_finished(reg, rid, timeout=3.0): + job = reg.get(rid) + if job is not None and job._thread is not None: + job._thread.join(timeout) + return reg.status(rid) + + +def test_finished_job_survives_restart(tmp_path): + store = tmp_path / "jobs.json" + r1 = JobRegistry(store_path=str(store)) + rid = r1.start("build", str(tmp_path), lambda: "the report") + assert _wait_finished(r1, rid)["status"] == "succeeded" + + r2 = JobRegistry(store_path=str(store)) # simulates a server restart + st = r2.status(rid) + assert st is not None + assert st["status"] == "succeeded" + assert st["result"] == "the report" + + +def test_running_job_loads_as_interrupted(tmp_path): + store = tmp_path / "jobs.json" + store.write_text( + json.dumps( + [ + { + "run_id": "abc", + "kind": "build", + "project_path": str(tmp_path), + "status": "running", + "result": None, + "error": None, + "started_at": "t0", + "ended_at": None, + "stop_requested": False, + } + ] + ) + ) + r = JobRegistry(store_path=str(store)) + assert r.status("abc")["status"] == "interrupted" + # An interrupted (not running) job no longer blocks a new job for that path. + rid = r.start("build", str(tmp_path), lambda: "ok") + assert rid + + +def test_no_store_path_is_ephemeral(tmp_path): + r = JobRegistry() # backward compatible: no persistence + rid = r.start("build", str(tmp_path), lambda: "x") + assert _wait_finished(r, rid)["status"] == "succeeded" + + +def test_corrupt_store_starts_empty(tmp_path): + store = tmp_path / "jobs.json" + store.write_text("{not valid json") + r = JobRegistry(store_path=str(store)) + assert r.list_jobs() == [] + + +def test_default_job_store_honors_env(monkeypatch, tmp_path): + from misterdev.core.execution.jobs import _default_job_store + + monkeypatch.setenv("MISTERDEV_STATE_DIR", str(tmp_path)) + assert _default_job_store() == str(tmp_path / "jobs.json") + + +def test_shared_registry_has_persistence_wired(): + from misterdev.core.execution.jobs import registry + + assert registry._store is not None # the MCP server's registry persists by default diff --git a/tests/test_jobs_progress.py b/tests/test_jobs_progress.py new file mode 100644 index 0000000..2cdab32 --- /dev/null +++ b/tests/test_jobs_progress.py @@ -0,0 +1,50 @@ +"""A2 — a job exposes task-level progress the running build reports. + +A target may accept a ``report(**fields)`` callback and push done/total/phase/message; +``status`` surfaces them. Legacy zero-arg targets keep working unchanged. +""" + +from misterdev.core.execution.jobs import JobRegistry + + +def _wait(reg, rid, timeout=3.0): + job = reg.get(rid) + if job is not None and job._thread is not None: + job._thread.join(timeout) + return reg.status(rid) + + +def test_status_reports_progress_from_target(tmp_path): + r = JobRegistry() + + def target(report): + report(done=1, total=3, phase="wave-1", message="task A") + return "done" + + rid = r.start("build", str(tmp_path), target) + st = _wait(r, rid) + assert st["status"] == "succeeded" + assert st["tasks_done"] == 1 and st["tasks_total"] == 3 + assert st["phase"] == "wave-1" and st["message"] == "task A" + + +def test_partial_progress_updates_only_given_fields(tmp_path): + r = JobRegistry() + + def target(report): + report(total=5) + report(done=2) # total must stay 5 + return "x" + + st = _wait(r, r.start("build", str(tmp_path), target)) + assert st["tasks_total"] == 5 and st["tasks_done"] == 2 + + +def test_update_progress_unknown_id_returns_false(): + assert JobRegistry().update_progress("nope", done=1) is False + + +def test_zero_arg_target_backward_compatible(tmp_path): + r = JobRegistry() + rid = r.start("build", str(tmp_path), lambda: "ok") + assert _wait(r, rid)["status"] == "succeeded" diff --git a/tests/test_judge_independence_wiring.py b/tests/test_judge_independence_wiring.py new file mode 100644 index 0000000..86c3a83 --- /dev/null +++ b/tests/test_judge_independence_wiring.py @@ -0,0 +1,61 @@ +"""R3 — the acceptance judge honors the same-model independence check (T1.2 residual). + +`GatesMixin._judge_generate` (the LLM PASS/FAIL acceptance judge — the highest-stakes +judge) called `generate_independent` directly, which routes through `with_model` +whenever a model is set. When `judge.model` equals the generator's model that is false +independence. Routing through the shared `build_independent_call` makes it detect and +skip the same-model case here too. +""" + +import contextlib +from types import SimpleNamespace + +from misterdev.task_executors.markdown_plan_executor.gates_mixin import GatesMixin + + +class _Ex(GatesMixin): + pass + + +class _Switchable: + def __init__(self, model="gen-model"): + self._model = model + self.entered = [] + + @property + def model(self): + return self._model + + def generate_code(self, prompt, system=""): + return "PASS" + + @contextlib.contextmanager + def with_model(self, m): + self.entered.append(m) + prev, self._model = self._model, m + try: + yield + finally: + self._model = prev + + +def _project(judge_model, client): + return SimpleNamespace(llm_client=client, config={"judge": {"model": judge_model}}) + + +def test_same_model_judge_is_not_routed(): + client = _Switchable("gen-model") + _Ex()._judge_generate(_project("gen-model", client), "prompt") + assert client.entered == [] + + +def test_distinct_model_judge_routes(): + client = _Switchable("gen-model") + out = _Ex()._judge_generate(_project("other-model", client), "prompt") + assert client.entered == ["other-model"] + assert out == "PASS" + + +def test_client_without_generate_code_returns_empty(): + project = SimpleNamespace(llm_client=object(), config={"judge": {}}) + assert _Ex()._judge_generate(project, "prompt") == "" diff --git a/tests/test_mcp.py b/tests/test_mcp.py index 046436b..75d82fb 100644 --- a/tests/test_mcp.py +++ b/tests/test_mcp.py @@ -278,13 +278,21 @@ def test_project_mcp_on_demand_manager_without_config(tmp_path): from misterdev.core.execution.project import Project proj = Project(tmp_path, {"name": "p"}) - assert proj.mcp is not None and proj.mcp.servers == [] + # The docs tool (context7/fetch) mounts by default (T2.4); the on-demand FIND + # loop can still mount more into the same manager. + assert proj.mcp is not None + assert {"context7", "fetch"} <= {s["name"] for s in proj.mcp.servers} isolated = Project(tmp_path, {"name": "p", "governance": {"network": "none"}}) - assert isolated.mcp is None + assert ( + isolated.mcp is None + ) # isolation refuses host provisioning (docs + on-demand) - off = Project(tmp_path, {"name": "p", "mcp": {"discover_on_demand": False}}) - assert off.mcp is None # nothing configured and on-demand explicitly off + off = Project( + tmp_path, + {"name": "p", "mcp": {"discover_on_demand": False, "docs_tool": False}}, + ) + assert off.mcp is None # fully opted out -> nothing to mount def test_project_mcp_built_from_config(tmp_path, server_path): diff --git a/tests/test_mcp_registry.py b/tests/test_mcp_registry.py index 95f9f21..e18a308 100644 --- a/tests/test_mcp_registry.py +++ b/tests/test_mcp_registry.py @@ -386,3 +386,35 @@ def test_discover_servers_routes_source_cgcone(monkeypatch): def test_fetch_cgcone_registry_never_raises(monkeypatch): monkeypatch.setattr(reg, "_http_get_json", lambda url, timeout: None) assert reg.fetch_cgcone_registry() == [] + + +def test_untrusted_runtime_hint_is_rejected(): + # A malicious registry entry cannot smuggle an arbitrary binary into `command` + # via runtimeHint; it falls back to the safe registry runtime (npx/uvx). + from misterdev.core.integration.mcp_registry import _to_config + + server = {"name": "evil/pkg"} + pkg = { + "registryType": "npm", + "identifier": "evilpkg", + "runtimeHint": "/bin/sh", + "transport": {"type": "stdio"}, + } + assert _to_config(server, pkg)["command"] == "npx" + + +def test_allowed_runtime_hint_is_honored(): + from misterdev.core.integration.mcp_registry import _to_config + + server = {"name": "ok/pkg"} + pkg = {"registryType": "npm", "identifier": "okpkg", "runtimeHint": "uvx"} + assert _to_config(server, pkg)["command"] == "uvx" + + +def test_absent_runtime_hint_uses_registry_runtime(): + from misterdev.core.integration.mcp_registry import _to_config + + assert ( + _to_config({"name": "p"}, {"registryType": "pypi", "identifier": "p"})["command"] + == "uvx" + ) diff --git a/tests/test_mcp_server.py b/tests/test_mcp_server.py index b6c1ed2..91913fd 100644 --- a/tests/test_mcp_server.py +++ b/tests/test_mcp_server.py @@ -1,16 +1,20 @@ """misterdev exposed as an MCP server — thin adapter over ProjectOrchestrator.""" import asyncio +import time from misterdev import mcp_server +from misterdev.core.execution.jobs import JobRegistry class _FakeOrch: last_build_succeeded = True calls: dict = {} - def build(self, path, args): - _FakeOrch.calls["build"] = (path, args) + def build(self, path, args, reference_dir=None, progress_cb=None): + _FakeOrch.calls["build"] = (path, args, reference_dir) + if progress_cb is not None: + progress_cb(done=1, total=1, phase="building") return "REPORT-BODY" def scan_directory(self, directory): @@ -22,21 +26,67 @@ def list_projects(self): def get_project_status(self, path): return {"path": path, "tasks": []} - def run_project(self, path, dry_run=False): + def run_project(self, path, dry_run=False, progress_cb=None): _FakeOrch.calls["run_project"] = (path, dry_run) + if progress_cb is not None: + progress_cb(done=2, total=4, phase="wave 1") def run_task(self, path, task_id): _FakeOrch.calls["run_task"] = (path, task_id) + def request_stop(self): + _FakeOrch.calls["request_stop"] = True + + def propose_plan(self, path, args): + _FakeOrch.calls["propose_plan"] = (path, args) + return {"items": [{"id": "P-001", "title": "t", "approved": False}]} + + def execute_plan(self, path, args): + _FakeOrch.calls["execute_plan"] = (path, args) + return "PLAN-EXEC-REPORT" + def _patch(monkeypatch): _FakeOrch.calls = {} monkeypatch.setattr(mcp_server, "ProjectOrchestrator", _FakeOrch) +def _patch_jobs(monkeypatch): + """Give the async tools a fresh, isolated registry per test.""" + reg = JobRegistry() + monkeypatch.setattr(mcp_server, "registry", reg) + return reg + + +def _await_status(reg, run_id, want, timeout=3.0): + deadline = time.monotonic() + timeout + while time.monotonic() < deadline: + state = reg.status(run_id) + if state and state["status"] == want: + return state + time.sleep(0.01) + raise AssertionError(f"{run_id} never reached {want}: {reg.status(run_id)}") + + def test_all_expected_tools_registered(): names = {t.name for t in asyncio.run(mcp_server.mcp.list_tools())} - assert {"scan", "list_projects", "status", "build", "run"} <= names + assert { + "scan", + "list_projects", + "status", + "build", + "run", + "report", + "build_async", + "run_async", + "job_status", + "stop_job", + "list_jobs", + "propose_plan", + "get_plan", + "approve_plan", + "execute_plan", + } <= names def test_tool_definitions_are_well_documented(): @@ -60,22 +110,30 @@ def test_build_routes_and_composes_flags(monkeypatch): out = mcp_server.build( "/repo", "add rate limiting", budget=5.0, parallel=True, max_tasks=3 ) - path, args = _FakeOrch.calls["build"] + path, args, reference_dir = _FakeOrch.calls["build"] assert path == "/repo" assert "add rate limiting" in args assert "--budget 5.0" in args assert "--parallel" in args assert "--max-tasks 3" in args + assert reference_dir is None assert "succeeded" in out and "REPORT-BODY" in out def test_build_dry_run_flag(monkeypatch): _patch(monkeypatch) mcp_server.build("/repo", "fix tests", dry_run=True) - _, args = _FakeOrch.calls["build"] + _, args, _ = _FakeOrch.calls["build"] assert "--dry-run" in args +def test_build_forwards_reference_dir(monkeypatch): + _patch(monkeypatch) + mcp_server.build("/repo", "port it", reference_dir="/donor/impl") + _, _, reference_dir = _FakeOrch.calls["build"] + assert reference_dir == "/donor/impl" + + def test_build_reports_failure(monkeypatch): _patch(monkeypatch) monkeypatch.setattr(_FakeOrch, "last_build_succeeded", False) @@ -97,3 +155,143 @@ def test_run_tool_routes(monkeypatch): assert _FakeOrch.calls["run_project"] == ("/repo", True) mcp_server.run("/repo", task_id="T-1") assert _FakeOrch.calls["run_task"] == ("/repo", "T-1") + + +def test_report_rejects_non_directory(): + out = mcp_server.report("/no/such/dir/xyz") + assert "error" in out + + +def test_report_reads_saved_artifacts(tmp_path): + reports = tmp_path / ".orchestrator" / "reports" + reports.mkdir(parents=True) + (reports / "report_20240101_000000.json").write_text( + '{"mode": "debug", "project": "p", "completed": ["T-1"], "failed": []}' + ) + out = mcp_server.report(str(tmp_path)) + assert out["latest_report"]["mode"] == "debug" + assert out["latest_report"]["completed"] == ["T-1"] + assert "audit" in out and "models" in out + + +def test_report_on_unbuilt_project_returns_null_report(tmp_path): + out = mcp_server.report(str(tmp_path)) + assert out["latest_report"] is None + assert "audit" in out and "models" in out + + +def test_build_async_starts_job_and_forwards_args(monkeypatch): + _patch(monkeypatch) + reg = _patch_jobs(monkeypatch) + out = mcp_server.build_async("/repo", "port it", budget=3.0, reference_dir="/ref") + assert out["status"] == "running" + run_id = out["run_id"] + _await_status(reg, run_id, "succeeded") + state = reg.status(run_id) + assert state["kind"] == "build" + assert state["result"] == "REPORT-BODY" + path, args, reference_dir = _FakeOrch.calls["build"] + assert path == "/repo" and reference_dir == "/ref" + assert "--budget 3.0" in args + + +def test_run_async_starts_job(monkeypatch): + _patch(monkeypatch) + reg = _patch_jobs(monkeypatch) + out = mcp_server.run_async("/repo") + _await_status(reg, out["run_id"], "succeeded") + assert _FakeOrch.calls["run_project"] == ("/repo", False) + + +def test_build_async_refuses_second_job_same_project(monkeypatch): + _patch(monkeypatch) + reg = _patch_jobs(monkeypatch) + hold = {"go": False} + + def _slow(self, path, args, reference_dir=None, progress_cb=None): + while not hold["go"]: + time.sleep(0.005) + return "REPORT-BODY" + + monkeypatch.setattr(_FakeOrch, "build", _slow) + first = mcp_server.build_async("/repo", "x") + assert "run_id" in first + second = mcp_server.build_async("/repo", "y") + assert "error" in second + hold["go"] = True + _await_status(reg, first["run_id"], "succeeded") + + +def test_job_status_and_list_and_stop(monkeypatch): + _patch(monkeypatch) + reg = _patch_jobs(monkeypatch) + out = mcp_server.build_async("/repo", "x") + run_id = out["run_id"] + _await_status(reg, run_id, "succeeded") + + assert mcp_server.job_status(run_id)["status"] == "succeeded" + assert mcp_server.job_status("nope") == {"error": "unknown run_id: nope"} + assert run_id in {j["run_id"] for j in mcp_server.list_jobs()["jobs"]} + # Stopping an already-finished job is a harmless no-op. + assert mcp_server.stop_job(run_id) == {"run_id": run_id, "stopping": False} + + +def test_propose_plan_routes(monkeypatch): + _patch(monkeypatch) + out = mcp_server.propose_plan("/repo", budget=4.0) + path, args = _FakeOrch.calls["propose_plan"] + assert path == "/repo" and "--budget 4.0" in args + assert out["items"][0]["id"] == "P-001" + + +def test_execute_plan_routes(monkeypatch): + _patch(monkeypatch) + out = mcp_server.execute_plan("/repo", budget=6.0) + path, args = _FakeOrch.calls["execute_plan"] + assert path == "/repo" and "--budget 6.0" in args + assert out == "PLAN-EXEC-REPORT" + + +def test_get_and_approve_plan_use_the_store(tmp_path): + from misterdev.core.planning import plan_store + + plan_store.save_plan( + tmp_path, + [{"title": "one"}, {"title": "two"}], + ) + # get_plan reads the persisted proposals. + got = mcp_server.get_plan(str(tmp_path)) + assert {it["id"] for it in got["items"]} == {"P-001", "P-002"} + # approve_plan flips the flag and persists. + approved = mcp_server.approve_plan(str(tmp_path), approve_ids=["P-002"]) + flags = {it["id"]: it["approved"] for it in approved["items"]} + assert flags == {"P-001": False, "P-002": True} + + +def test_approve_plan_without_plan_errors(tmp_path): + out = mcp_server.approve_plan(str(tmp_path), approve_all=True) + assert "error" in out + + +def test_get_plan_empty_when_none(tmp_path): + assert mcp_server.get_plan(str(tmp_path)) == {"items": []} + + +def test_run_async_reports_task_progress(monkeypatch): + _patch(monkeypatch) + reg = _patch_jobs(monkeypatch) + run_id = mcp_server.run_async("/repo")["run_id"] + st = _await_status(reg, run_id, "succeeded") + # Progress reported by the run flowed through the reporter into job status. + assert st["tasks_done"] == 2 and st["tasks_total"] == 4 + assert st["phase"] == "wave 1" + + +def test_build_async_reports_task_progress(monkeypatch): + _patch(monkeypatch) + reg = _patch_jobs(monkeypatch) + run_id = mcp_server.build_async("/repo", "do it", budget=1.0)["run_id"] + st = _await_status(reg, run_id, "succeeded") + assert ( + st["tasks_done"] == 1 and st["tasks_total"] == 1 and st["phase"] == "building" + ) diff --git a/tests/test_model_selector.py b/tests/test_model_selector.py index 3bc0491..9534642 100644 --- a/tests/test_model_selector.py +++ b/tests/test_model_selector.py @@ -341,3 +341,22 @@ def test_escalation_climbs_one_rung_per_attempt_when_exploring(ledger): assert sel.select("feature", "large", 0, 4) == "v/cheap" assert sel.select("feature", "large", 1, 4) == "v/mid" assert sel.select("feature", "large", 2, 4) == "v/top" + + +def test_final_attempt_widens_when_strongest_tier_all_incompetent(ledger, monkeypatch): + sel = ModelSelector( + _config(models={"cheap": "openai/small", "strong": "anthropic/big"}), ledger + ) + # The strongest tier's only model is proven incompetent for this cell; the final + # attempt must NOT silently fall back to the client default — it widens to the + # best usable model across all tiers. + monkeypatch.setattr(sel, "_incompetent", lambda m, c, x: m == "anthropic/big") + assert sel.select("feature", "medium", 2, 3) == "openai/small" + + +def test_final_attempt_none_only_when_everything_incompetent(ledger, monkeypatch): + sel = ModelSelector( + _config(models={"cheap": "openai/small", "strong": "anthropic/big"}), ledger + ) + monkeypatch.setattr(sel, "_incompetent", lambda m, c, x: True) + assert sel.select("feature", "medium", 2, 3) is None diff --git a/tests/test_mutation_gate_changed_region.py b/tests/test_mutation_gate_changed_region.py new file mode 100644 index 0000000..c2eaeba --- /dev/null +++ b/tests/test_mutation_gate_changed_region.py @@ -0,0 +1,66 @@ +"""T1.3 — the changed-region mutation gate scores ALL changed source files. + +Previously the check scored only the LARGEST edited source file and returned, so a +multi-file fix whose crux is a small file was never mutation-tested (a weak suite +there went unsurfaced). This asserts every changed non-test source file with a +mutable region is scored, while test/whitespace files are still skipped. +""" + +from types import SimpleNamespace + +from misterdev.task_executors.markdown_plan_executor.edits_mixin import EditsMixin + + +class _Ex(EditsMixin): + pass + + +def _project(tmp_path): + return SimpleNamespace( + path=tmp_path, + config={ + "orchestrator": {"changed_region_mutation": True}, + "mutation": {}, + "test_command": "pytest", + }, + ) + + +def test_all_changed_source_files_are_scored(tmp_path, monkeypatch): + (tmp_path / "big.py").write_text("def a():\n return 1\n" * 5) + (tmp_path / "small.py").write_text("def b():\n return 2\n") + pre_edit = {"big.py": "old big", "small.py": "old small"} + + scored = [] + + def _fake_run(project_path, path, old, new, cmd, runner, min_score, max_mutants): + scored.append(path) + return SimpleNamespace(status="scored", reason="ok") + + monkeypatch.setattr( + "misterdev.core.verification.changed_region_mutation.run_changed_region_mutation", + _fake_run, + ) + _Ex()._changed_region_mutation_check( + _project(tmp_path), pre_edit, "pytest", tmp_path + ) + assert set(scored) == {"big.py", "small.py"} + + +def test_test_files_are_not_scored(tmp_path, monkeypatch): + (tmp_path / "src.py").write_text("def a():\n return 1\n") + (tmp_path / "test_src.py").write_text("def test_a():\n assert True\n") + pre_edit = {"src.py": "old", "test_src.py": "old"} + + scored = [] + monkeypatch.setattr( + "misterdev.core.verification.changed_region_mutation.run_changed_region_mutation", + lambda *a, **k: ( + scored.append(a[1]), + SimpleNamespace(status="scored", reason="ok"), + )[1], + ) + _Ex()._changed_region_mutation_check( + _project(tmp_path), pre_edit, "pytest", tmp_path + ) + assert scored == ["src.py"] diff --git a/tests/test_no_truncate_edit_region.py b/tests/test_no_truncate_edit_region.py new file mode 100644 index 0000000..8dc1db8 --- /dev/null +++ b/tests/test_no_truncate_edit_region.py @@ -0,0 +1,40 @@ +"""T2.3 — the edit region is never truncated by the context budget. + +The edit region (code_context) holds the exact verbatim lines a SEARCH/REPLACE edit +must match. Under budget pressure the allocator would trim it (priority 1 is trimmed +last, but still trimmed), leaving the model to edit blind against a tail it can no +longer see. A section registered `truncatable=False` must be kept verbatim even when +it alone overflows the budget; ordinary sections still truncate. +""" + +from misterdev.core.economics.context_budget import ContextBudget + +# An edit region that ALONE overflows the tiny budget below. +EDIT = "\n".join( + f"{i:04d}: exact source line that must survive verbatim" for i in range(80) +) +FILLER = "\n".join(f"filler note {i}" for i in range(40)) + + +def test_non_truncatable_edit_region_kept_verbatim_under_pressure(): + budget = ContextBudget(max_tokens=200, reserved_tokens=50) # available=150 + budget.set("code_context", EDIT, priority=1, truncatable=False) + budget.set("scratchpad", FILLER, priority=3) + out = budget.allocate() + assert out["code_context"] == EDIT + assert "omitted" not in out["code_context"] + + +def test_ordinary_section_still_truncates(): + budget = ContextBudget(max_tokens=200, reserved_tokens=50) + budget.set("code_context", EDIT, priority=1) # truncatable by default + budget.set("scratchpad", FILLER, priority=3) + out = budget.allocate() + assert "omitted" in out["code_context"] + + +def test_no_pressure_returns_everything_verbatim(): + budget = ContextBudget(max_tokens=100000) + budget.set("code_context", EDIT, priority=1, truncatable=False) + out = budget.allocate() + assert out["code_context"] == EDIT diff --git a/tests/test_orchestrator_config.py b/tests/test_orchestrator_config.py index b59398a..140e08d 100644 --- a/tests/test_orchestrator_config.py +++ b/tests/test_orchestrator_config.py @@ -139,7 +139,7 @@ def test_uses_configured_max_workers(self): tasks = self._make_tasks(3) # 3 tasks, max_workers=8 → min(3,8)=3 with patch( - "misterdev.agent.concurrent.futures.ThreadPoolExecutor" + "misterdev.core.execution.parallel.concurrent.futures.ThreadPoolExecutor" ) as MockPool: mock_ctx = MagicMock() MockPool.return_value.__enter__ = MagicMock(return_value=mock_ctx) @@ -147,7 +147,7 @@ def test_uses_configured_max_workers(self): mock_ctx.submit.return_value = MagicMock() with patch( - "misterdev.agent.concurrent.futures.as_completed", + "misterdev.core.execution.parallel.concurrent.futures.as_completed", return_value=[], ): orchestrator._execute_parallel(tasks, executor, project) @@ -162,7 +162,7 @@ def test_caps_at_task_count(self): tasks = self._make_tasks(2) with patch( - "misterdev.agent.concurrent.futures.ThreadPoolExecutor" + "misterdev.core.execution.parallel.concurrent.futures.ThreadPoolExecutor" ) as MockPool: mock_ctx = MagicMock() MockPool.return_value.__enter__ = MagicMock(return_value=mock_ctx) @@ -170,7 +170,7 @@ def test_caps_at_task_count(self): mock_ctx.submit.return_value = MagicMock() with patch( - "misterdev.agent.concurrent.futures.as_completed", + "misterdev.core.execution.parallel.concurrent.futures.as_completed", return_value=[], ): orchestrator._execute_parallel(tasks, executor, project) @@ -186,7 +186,7 @@ def test_default_max_workers_when_no_orchestrator_config(self): tasks = self._make_tasks(6) # 6 tasks, default max_workers=4 → min(6,4)=4 with patch( - "misterdev.agent.concurrent.futures.ThreadPoolExecutor" + "misterdev.core.execution.parallel.concurrent.futures.ThreadPoolExecutor" ) as MockPool: mock_ctx = MagicMock() MockPool.return_value.__enter__ = MagicMock(return_value=mock_ctx) @@ -194,7 +194,7 @@ def test_default_max_workers_when_no_orchestrator_config(self): mock_ctx.submit.return_value = MagicMock() with patch( - "misterdev.agent.concurrent.futures.as_completed", + "misterdev.core.execution.parallel.concurrent.futures.as_completed", return_value=[], ): orchestrator._execute_parallel(tasks, executor, project) @@ -209,7 +209,7 @@ def test_default_max_workers_when_orchestrator_config_empty(self): tasks = self._make_tasks(10) # 10 tasks, default max_workers=4 → min(10,4)=4 with patch( - "misterdev.agent.concurrent.futures.ThreadPoolExecutor" + "misterdev.core.execution.parallel.concurrent.futures.ThreadPoolExecutor" ) as MockPool: mock_ctx = MagicMock() MockPool.return_value.__enter__ = MagicMock(return_value=mock_ctx) @@ -217,7 +217,7 @@ def test_default_max_workers_when_orchestrator_config_empty(self): mock_ctx.submit.return_value = MagicMock() with patch( - "misterdev.agent.concurrent.futures.as_completed", + "misterdev.core.execution.parallel.concurrent.futures.as_completed", return_value=[], ): orchestrator._execute_parallel(tasks, executor, project) @@ -276,7 +276,7 @@ def fake_submit(fn, task, proj, use_git_branch=False): return fut with patch( - "misterdev.agent.concurrent.futures.ThreadPoolExecutor" + "misterdev.core.execution.parallel.concurrent.futures.ThreadPoolExecutor" ) as MockPool: mock_ctx = MagicMock() MockPool.return_value.__enter__ = MagicMock(return_value=mock_ctx) @@ -284,7 +284,7 @@ def fake_submit(fn, task, proj, use_git_branch=False): mock_ctx.submit.side_effect = fake_submit with patch( - "misterdev.agent.concurrent.futures.as_completed", + "misterdev.core.execution.parallel.concurrent.futures.as_completed", side_effect=lambda d: list(d), ): results = orchestrator._execute_parallel(tasks, executor, project) diff --git a/tests/test_orchestrator_fixes.py b/tests/test_orchestrator_fixes.py index 7d3ef4c..d01cddf 100644 --- a/tests/test_orchestrator_fixes.py +++ b/tests/test_orchestrator_fixes.py @@ -445,6 +445,497 @@ def execute(self, task, proj, use_git_branch=True): assert not any(wt_root.iterdir()) if wt_root.exists() else True +def test_execute_parallel_worktrees_tolerates_stale_task_branch(): + """Regression: a leftover ``task/`` branch from a prior failed run must not + collide with this run's worktree creation, and no throwaway branch is left + behind (whether the task succeeds or fails).""" + from unittest.mock import MagicMock + import misterdev.agent as agent_mod + + def _branches(path): + out = subprocess.run( + ["git", "branch", "--format=%(refname:short)"], + cwd=path, + capture_output=True, + text=True, + ).stdout + return [b.strip() for b in out.splitlines() if b.strip()] + + with tempfile.TemporaryDirectory() as td: + td = Path(td) + _git(td, "init") + _git(td, "config", "user.email", "t@t.t") + _git(td, "config", "user.name", "t") + (td / "base.txt").write_text("base\n") + _git(td, "add", "-A") + _git(td, "commit", "-m", "init") + # Two stale leftover branches from an imagined prior run — the exact names + # the old fixed-name scheme would try to re-create and fail on. + _git(td, "branch", "task/T-A") + _git(td, "branch", "task/T-B") + + project = MagicMock() + project.path = td + project.config = { + "orchestrator": {"parallel_mode": "worktree", "max_workers": 2} + } + + class FakeExec: + def execute(self, task, proj, use_git_branch=True): + if task.id == "T-B": # this one fails: it must still clean up + raise RuntimeError("boom") + f = Path(proj.path) / f"{task.id}.txt" + f.write_text(task.id) + _git(proj.path, "add", "-A") + _git(proj.path, "commit", "-m", f"task({task.id}): work") + r = MagicMock() + r.status = "completed" + return r + + orch = agent_mod.ProjectOrchestrator() + results = orch._execute_parallel_worktrees( + [_mock_task("T-A"), _mock_task("T-B")], FakeExec(), project + ) + + by_id = {t.id: (r, e) for t, r, e in results} + # The stale branch did not block T-A; its work merged despite the collision. + assert by_id["T-A"][0] is not None and by_id["T-A"][0].status == "completed" + assert (td / "T-A.txt").exists() + assert by_id["T-B"][1] is not None # T-B surfaced its error, didn't wedge + # No run-created throwaway branches remain (only the two inert stale names). + assert sorted(_branches(td)) == ["main", "task/T-A", "task/T-B"] or sorted( + _branches(td) + ) == ["master", "task/T-A", "task/T-B"] + + +def test_worktree_setup_command_detection(tmp_path): + """Priming command: explicit config wins ('' disables); else auto-detect from + the lockfile so a gate never pays a full install inside its own timeout.""" + from unittest.mock import MagicMock + import misterdev.agent as agent_mod + + orch = agent_mod.ProjectOrchestrator() + + def proj(sub, files=(), cfg=None): + d = tmp_path / sub + d.mkdir() + for f in files: + (d / f).write_text("") + p = MagicMock() + p.path = d + p.config = cfg or {} + return p + + explicit = proj("a", cfg={"orchestrator": {"worktree_setup_command": "make deps"}}) + assert orch._worktree_setup_command(explicit) == "make deps" + disabled = proj("b", cfg={"orchestrator": {"worktree_setup_command": ""}}) + assert orch._worktree_setup_command(disabled) is None + assert "pnpm install" in orch._worktree_setup_command(proj("c", ["pnpm-lock.yaml"])) + assert orch._worktree_setup_command(proj("d", ["package-lock.json"])) == "npm ci" + assert "yarn install" in orch._worktree_setup_command(proj("e", ["yarn.lock"])) + assert orch._worktree_setup_command(proj("f")) is None # nothing to install + + +def test_execute_parallel_worktrees_primes_deps_before_gate(): + """The setup command runs INSIDE each worktree before the task body, so the + gate finds dependencies already present instead of installing on the hot path.""" + from unittest.mock import MagicMock + import misterdev.agent as agent_mod + + with tempfile.TemporaryDirectory() as td: + td = Path(td) + _git(td, "init") + _git(td, "config", "user.email", "t@t.t") + _git(td, "config", "user.name", "t") + (td / "base.txt").write_text("base\n") + _git(td, "add", "-A") + _git(td, "commit", "-m", "init") + + project = MagicMock() + project.path = td + project.config = { + "orchestrator": { + "parallel_mode": "worktree", + "max_workers": 2, + # a marker-writing stand-in for `pnpm install` + "worktree_setup_command": "touch .primed", + } + } + + seen = {} + + class FakeExec: + def execute(self, task, proj, use_git_branch=True): + # The prime step must have already run in this worktree. + seen[task.id] = (Path(proj.path) / ".primed").exists() + r = MagicMock() + r.status = "completed" + return r + + orch = agent_mod.ProjectOrchestrator() + orch._execute_parallel_worktrees( + [_mock_task("T-A"), _mock_task("T-B")], FakeExec(), project + ) + assert seen == {"T-A": True, "T-B": True} + + +def _worktree_repo(td: Path): + """A minimal committed git repo usable as a parallel-worktree base.""" + _git(td, "init") + _git(td, "config", "user.email", "t@t.t") + _git(td, "config", "user.name", "t") + (td / "base.txt").write_text("base\n") + _git(td, "add", "-A") + _git(td, "commit", "-m", "init") + + +def test_worktree_healthcheck_probes_after_prime(): + """The health probe runs INSIDE each primed worktree; a passing probe lets the + task proceed and leaves its marker behind.""" + from unittest.mock import MagicMock + import misterdev.agent as agent_mod + + with tempfile.TemporaryDirectory() as td: + td = Path(td) + _worktree_repo(td) + project = MagicMock() + project.path = td + project.config = { + "orchestrator": { + "parallel_mode": "worktree", + "max_workers": 2, + "worktree_setup_command": "touch .primed", + # passes only if priming ran first, and records that it ran + "worktree_healthcheck_command": "test -f .primed && touch .probed", + } + } + + seen = {} + + class FakeExec: + def execute(self, task, proj, use_git_branch=True): + seen[task.id] = (Path(proj.path) / ".probed").exists() + r = MagicMock() + r.status = "completed" + return r + + agent_mod.ProjectOrchestrator()._execute_parallel_worktrees( + [_mock_task("T-A")], FakeExec(), project + ) + assert seen == {"T-A": True} # probe ran (and passed) before the task body + + +def test_worktree_healthcheck_reprimes_once_on_failure(): + """A failing probe re-primes the deps exactly once and re-probes; a probe that + then passes lets the task run without flagging the environment.""" + from unittest.mock import MagicMock + import misterdev.agent as agent_mod + + with tempfile.TemporaryDirectory() as td: + td = Path(td) + _worktree_repo(td) + project = MagicMock() + project.path = td + project.config = { + "orchestrator": { + "parallel_mode": "worktree", + "max_workers": 1, + # each prime appends one line; the probe passes only after TWO primes + "worktree_setup_command": "echo x >> .primes", + "worktree_healthcheck_command": "test $(wc -l < .primes) -ge 2", + } + } + + primes = {} + + class FakeExec: + def execute(self, task, proj, use_git_branch=True): + primes[task.id] = (Path(proj.path) / ".primes").read_text().count("x") + r = MagicMock() + r.status = "completed" + return r + + agent_mod.ProjectOrchestrator()._execute_parallel_worktrees( + [_mock_task("T-A")], FakeExec(), project + ) + # Primed exactly twice: the initial prime plus one re-prime after the + # probe failed. A third prime would mean the "once" bound was violated. + assert primes == {"T-A": 2} + + +def test_worktree_healthcheck_flags_unhealthy_environment(caplog): + """A probe that stays red after the single re-prime logs the worktree as an + ENVIRONMENT fault (not the code), and does not re-prime more than once.""" + import logging + from unittest.mock import MagicMock + import misterdev.agent as agent_mod + + with tempfile.TemporaryDirectory() as td: + td = Path(td) + _worktree_repo(td) + project = MagicMock() + project.path = td + project.config = { + "orchestrator": { + "parallel_mode": "worktree", + "max_workers": 1, + "worktree_setup_command": "echo x >> .primes", + "worktree_healthcheck_command": "false", # never resolves + } + } + + class FakeExec: + def execute(self, task, proj, use_git_branch=True): + r = MagicMock() + r.status = "completed" + return r + + with caplog.at_level(logging.WARNING, logger="project_orchestrator"): + agent_mod.ProjectOrchestrator()._execute_parallel_worktrees( + [_mock_task("T-A")], FakeExec(), project + ) + # Exactly one re-prime attempt (the "once" bound) and one unhealthy report. + assert caplog.text.count("re-priming deps once") == 1 + assert caplog.text.count("ENVIRONMENT unhealthy for T-A") == 1 + + +def test_post_merge_healthcheck_reverts_broken_base_keeps_clean(): + """A merged change that fails the base-branch gate is rolled back (base tree + restored, task returned unfinished); a clean merge is kept.""" + import subprocess + from unittest.mock import MagicMock + import misterdev.agent as agent_mod + + with tempfile.TemporaryDirectory() as td: + td = Path(td) + _worktree_repo(td) + project = MagicMock() + project.path = td + project.config = { + "orchestrator": {"parallel_mode": "worktree", "max_workers": 1}, + # top-level gate (no targets): the base is broken iff `.broken` exists + "typecheck_command": "test ! -f .broken", + } + + class FakeExec: + def execute(self, task, proj, use_git_branch=True): + wt = Path(proj.path) + if task.id == "T-break": + (wt / ".broken").write_text("boom\n") + else: + (wt / f"ok-{task.id}.txt").write_text("ok\n") + subprocess.run(["git", "add", "-A"], cwd=wt, check=True) + subprocess.run( + ["git", "commit", "-m", f"task {task.id}"], + cwd=wt, + check=True, + capture_output=True, + ) + r = MagicMock() + r.status = "completed" + return r + + def _run_command(self, project, command, timeout=120, cwd=None): + p = subprocess.run( + command, + shell=True, + cwd=cwd or project.path, + capture_output=True, + text=True, + ) + return p.returncode == 0, (p.stdout or "") + (p.stderr or "") + + results = agent_mod.ProjectOrchestrator()._execute_parallel_worktrees( + [_mock_task("T-break"), _mock_task("T-ok")], FakeExec(), project + ) + + # Base branch is NOT left broken: the breaking merge was rolled back... + assert not (td / ".broken").exists() + # ...and the clean merge was kept. + assert (td / "ok-T-ok.txt").exists() + + by_id = {t.id: (res, err) for t, res, err in results} + # The broken task is returned unfinished (no completed result, error set). + assert by_id["T-break"][0] is None and by_id["T-break"][1] is not None + # The clean task stays completed. + assert getattr(by_id["T-ok"][0], "status", None) == "completed" + + +def test_post_merge_healthcheck_disabled_keeps_broken_merge(): + """With orchestrator.post_merge_healthcheck false, a merge is never gated, so a + base-breaking change is kept (the opt-out path).""" + import subprocess + from unittest.mock import MagicMock + import misterdev.agent as agent_mod + + with tempfile.TemporaryDirectory() as td: + td = Path(td) + _worktree_repo(td) + project = MagicMock() + project.path = td + project.config = { + "orchestrator": { + "parallel_mode": "worktree", + "max_workers": 1, + "post_merge_healthcheck": False, + }, + "typecheck_command": "test ! -f .broken", + } + + class FakeExec: + def execute(self, task, proj, use_git_branch=True): + (Path(proj.path) / ".broken").write_text("boom\n") + subprocess.run(["git", "add", "-A"], cwd=proj.path, check=True) + subprocess.run( + ["git", "commit", "-m", "break"], + cwd=proj.path, + check=True, + capture_output=True, + ) + r = MagicMock() + r.status = "completed" + return r + + def _run_command(self, *a, **k): # gate must not be consulted + raise AssertionError("post-merge gate ran while disabled") + + agent_mod.ProjectOrchestrator()._execute_parallel_worktrees( + [_mock_task("T-break")], FakeExec(), project + ) + assert (td / ".broken").exists() # merge kept, no gate, no revert + + +class _SharedFileExec: + """FakeExec whose tasks all append to the SAME file and commit in their + worktree — so two run in parallel would race and conflict on merge.""" + + def execute(self, task, proj, use_git_branch=True): + import subprocess + from unittest.mock import MagicMock + + wt = Path(proj.path) + f = wt / "shared.txt" + f.write_text(f.read_text() + f"{task.id}\n") + subprocess.run(["git", "add", "-A"], cwd=wt, check=True) + subprocess.run( + ["git", "commit", "-m", f"task {task.id}"], + cwd=wt, + check=True, + capture_output=True, + ) + r = MagicMock() + r.status = "completed" + return r + + +def _shared_file_repo(td: Path): + _git(td, "init") + _git(td, "config", "user.email", "t@t.t") + _git(td, "config", "user.name", "t") + (td / "shared.txt").write_text("line0\n") + _git(td, "add", "-A") + _git(td, "commit", "-m", "init") + + +def test_conflicting_wave_serialized_completes_without_conflict(): + """Two tasks that DECLARE the same file are split into separate sub-waves, so + the second builds on the first's merge and neither conflicts — both complete + and the shared file carries both edits.""" + from unittest.mock import MagicMock + import misterdev.agent as agent_mod + + with tempfile.TemporaryDirectory() as td: + td = Path(td) + _shared_file_repo(td) + project = MagicMock() + project.path = td + project.config = { + "orchestrator": {"parallel_mode": "worktree", "max_workers": 4} + } + + a, b = _mock_task("T-A"), _mock_task("T-B") + a.files_to_modify = ["shared.txt"] + b.files_to_modify = ["shared.txt"] + + results = agent_mod.ProjectOrchestrator()._execute_parallel_worktrees( + [a, b], _SharedFileExec(), project + ) + by_id = {t.id: (res, err) for t, res, err in results} + assert getattr(by_id["T-A"][0], "status", None) == "completed" + assert getattr(by_id["T-B"][0], "status", None) == "completed" + # Both edits landed serially on the shared file — no conflict, no clobber. + text = (td / "shared.txt").read_text() + assert "T-A" in text and "T-B" in text + assert "<<<<<<<" not in text # no conflict markers + + +def test_unserialized_conflict_aborts_cleanly_and_requeues(): + """With serialization off, two tasks editing the same file race: one merges, + the other conflicts. The conflicted merge is ABORTED (base left clean, not + mid-merge) and that task is returned unfinished for re-queue.""" + import subprocess + from unittest.mock import MagicMock + import misterdev.agent as agent_mod + + with tempfile.TemporaryDirectory() as td: + td = Path(td) + _shared_file_repo(td) + project = MagicMock() + project.path = td + project.config = { + "orchestrator": { + "parallel_mode": "worktree", + "max_workers": 4, + "serialize_conflicting_tasks": False, # force the race + } + } + + a, b = _mock_task("T-A"), _mock_task("T-B") + a.files_to_modify = ["shared.txt"] + b.files_to_modify = ["shared.txt"] + + results = agent_mod.ProjectOrchestrator()._execute_parallel_worktrees( + [a, b], _SharedFileExec(), project + ) + statuses = [getattr(res, "status", None) for _t, res, _e in results] + errors = [err for _t, _res, err in results] + # Exactly one merged; the other conflicted and was returned unfinished. + assert statuses.count("completed") == 1 + assert sum(1 for e in errors if e is not None) == 1 + + # The base is CLEAN: the conflicted merge was aborted, not left mid-merge. + assert not (td / ".git" / "MERGE_HEAD").exists() + porcelain = subprocess.run( + ["git", "status", "--porcelain"], cwd=td, capture_output=True, text=True + ).stdout + assert porcelain.strip() == "" # no conflict markers / unmerged paths + assert "<<<<<<<" not in (td / "shared.txt").read_text() + + +def test_worktree_healthcheck_command_resolution(tmp_path: Path): + """Explicit config wins and "" disables; otherwise a node project auto-detects + a toolchain probe (tsc when TS is used, else a dependency resolve) and a + non-node project has no probe.""" + from misterdev.agent_helpers import worktree_healthcheck_command + + assert worktree_healthcheck_command({}, tmp_path) is None # non-node: no probe + cfg = {"orchestrator": {"worktree_healthcheck_command": "make check"}} + assert worktree_healthcheck_command(cfg, tmp_path) == "make check" + off = {"orchestrator": {"worktree_healthcheck_command": ""}} + (tmp_path / "package.json").write_text('{"dependencies":{"hono":"^4"}}') + assert worktree_healthcheck_command(off, tmp_path) is None # "" disables + # package.json with a dependency but no tsconfig -> resolve the first dep. + assert ( + worktree_healthcheck_command({}, tmp_path) + == "node -e \"require.resolve('hono')\"" + ) + # tsconfig.json present -> prefer the TypeScript toolchain probe. + (tmp_path / "tsconfig.json").write_text("{}") + assert ( + worktree_healthcheck_command({}, tmp_path) == "npx --no-install tsc --version" + ) + + # --- streaming with early abort (028) --------------------------------------- @@ -955,6 +1446,63 @@ def test_run_project_executes_in_dependency_order(): assert executed == ["001-a", "002-b"] +def test_run_project_writes_classified_run_summary(): + """The `run --tasks` path now emits run_summary.json with the failure taxonomy + (it previously had no summary at all), so a run is diagnosable at a glance.""" + import json + from unittest.mock import patch, MagicMock + import misterdev.agent as agent_mod + from misterdev.utils.file_utils import orchestrator_state_file + + with tempfile.TemporaryDirectory() as td: + project = MagicMock() + project.name = "p" + project.path = Path(td) + project.env_manager = None + project.config = { + "language": "python", + "orchestrator": {"max_consecutive_failures": 3}, + } + done, parked = _mock_task("T-done"), _mock_task("T-park") + project.task_manager.get_pending_tasks.return_value = [done, parked] + + def _exec(task, proj, *a, **k): + r = MagicMock() + if task.id == "T-park": + r.status = "deferred" + r.message = "could not complete after all attempts" + r.questions = [] + else: + r.status = "completed" + return r + + with ( + patch.object( + agent_mod.ProjectOrchestrator, "_get_or_register", return_value=project + ), + patch("misterdev.agent.topological_sort", side_effect=lambda x: x), + patch("misterdev.agent.Scratchpad"), + patch("misterdev.agent.ContractRegistry"), + patch("misterdev.agent.ChangeTracker"), + patch("misterdev.agent.StrategyOptimizer") as MockStrat, + patch("misterdev.agent.ProgressTracker") as MockProg, + patch("misterdev.agent.MarkdownPlanExecutor") as MockExec, + ): + MockProg.return_value.completed = [] + MockProg.return_value.is_done.return_value = False + MockStrat.return_value.select_best_strategy.return_value = "iterative" + MockExec.return_value.execute.side_effect = _exec + agent_mod.ProjectOrchestrator().run_project(td) + + data = json.loads( + orchestrator_state_file(Path(td), "run_summary.json").read_text() + ) + assert data["completed"] == 1 + assert data["deferred"] == 1 + assert data["failure_breakdown"] == {"deferred-needs-input": 1} + assert data["top_obstacle"] == "deferred-needs-input" + + # --- budget kill-switch / graceful checkpointing ---------------------------- @@ -1063,6 +1611,400 @@ def _run_command(self, *a, **k): assert any("per-task cost cap" in d for d in report.key_decisions) +class _SkipExec: + """Executor stand-in that records every task it is asked to run, so a test can + assert an already-satisfied task never reaches execution (no worktree).""" + + executed: list + + def __init__(self, *a, **k): + pass + + def execute(self, t, proj, *a, **k): + from unittest.mock import MagicMock + + type(self).executed.append(t.id) + r = MagicMock() + r.status = "completed" + return r + + def _run_command(self, *a, **k): + return True, "" + + def find_task_commit(self, *a, **k): + return None + + +def _run_execute_tasks(project, tasks): + from unittest.mock import patch + import misterdev.agent as agent_mod + from misterdev.core.modes import BuildFlags + + _SkipExec.executed = [] + # Patch the peripheral collaborators (as the other _execute_tasks tests do), + # but keep a REAL ProgressTracker so it loads the on-disk ledger this test + # pre-populated — that ledger is what the skip decision reads. + with ( + patch.object(agent_mod, "MarkdownPlanExecutor", _SkipExec), + patch("misterdev.agent.Scratchpad"), + patch("misterdev.agent.RealTimeAligner"), + patch("misterdev.agent.ContractRegistry"), + patch("misterdev.agent.ChangeTracker"), + patch("misterdev.agent.StrategyOptimizer") as MockStrat, + ): + MockStrat.return_value.select_best_strategy.return_value = "iterative" + orch = agent_mod.ProjectOrchestrator() + report = _fresh_report() + orch._execute_tasks(tasks, project, BuildFlags(no_rollback=True), report) + return report, list(_SkipExec.executed) + + +def test_satisfied_task_skipped_before_worktree(): + """A ready task whose content hash matches its recorded completion is marked + done WITHOUT ever reaching the executor (so no worktree is spawned).""" + from misterdev.core.execution.progress import ProgressTracker, compute_task_hash + + with tempfile.TemporaryDirectory() as td: + project = _budget_project(td) + task = _mock_task("T-done") + task.acceptance_criteria = "" # a real str -> a deterministic hash + # Record a prior completion at the task's CURRENT content hash. + ProgressTracker(Path(td)).mark_completed( + task.id, compute_task_hash(task, Path(td)) + ) + + report, executed = _run_execute_tasks(project, [task]) + + assert executed == [] # never dispatched -> no worktree + assert task in report.completed_tasks + + +def test_changed_or_absent_hash_still_runs(): + """A recorded completion at a STALE hash, and a task with no recorded + completion at all, both re-run — the skip keys on the content hash, not the id + alone.""" + from misterdev.core.execution.progress import ProgressTracker + + with tempfile.TemporaryDirectory() as td: + project = _budget_project(td) + stale = _mock_task("T-stale") + stale.acceptance_criteria = "" + absent = _mock_task("T-absent") + absent.acceptance_criteria = "" + # 'stale' is recorded completed but at a hash that no longer matches; + # 'absent' has no record. Neither may be skipped. + ProgressTracker(Path(td)).mark_completed("T-stale", "0000stalehash000") + + _, executed = _run_execute_tasks(project, [stale, absent]) + + assert set(executed) == {"T-stale", "T-absent"} + + +def test_skip_satisfied_tasks_flag_off_forces_rerun(): + """With orchestrator.skip_satisfied_tasks false, even a hash-matched completion + re-runs (the skip is opt-out).""" + from misterdev.core.execution.progress import ProgressTracker, compute_task_hash + + with tempfile.TemporaryDirectory() as td: + project = _budget_project(td) + project.config["orchestrator"]["skip_satisfied_tasks"] = False + task = _mock_task("T-done") + task.acceptance_criteria = "" + ProgressTracker(Path(td)).mark_completed( + task.id, compute_task_hash(task, Path(td)) + ) + + _, executed = _run_execute_tasks(project, [task]) + + assert executed == ["T-done"] # flag off -> re-run despite the match + + +def test_wave_infra_count_counts_only_unrecovered_infra_failures(): + from unittest.mock import MagicMock + import misterdev.agent as agent_mod + + def _res(status, logs=""): + r = MagicMock() + r.status = status + r.logs = logs + r.message = "" + return r + + results = [ + (_mock_task("A"), _res("failed", "Command timed out after 120s"), None), + (_mock_task("B"), _res("completed", "Command timed out after 120s"), None), + (_mock_task("C"), _res("failed", "AssertionError: 1 != 2"), None), + (_mock_task("D"), None, RuntimeError("waiting for the lock on the store")), + ] + # A (infra, failed) and D (infra, in the raised error) count; B completed + # (self-healed) and C (a real code error) do not. + assert agent_mod.ProjectOrchestrator._wave_infra_count(results) == 2 + + +def test_apply_wave_tuning_scales_config_from_base(): + from unittest.mock import MagicMock + import misterdev.agent as agent_mod + from misterdev.core.execution.adaptive import WaveTuning + + project = MagicMock() + project.config = {"orchestrator": {}, "build": {}} + base = {"workers": 8, "setup": 600, "build": 120, "test": 180} + agent_mod.ProjectOrchestrator()._apply_wave_tuning( + project, WaveTuning(2, 2.0), base + ) + assert project.config["orchestrator"]["max_workers"] == 2 + assert project.config["orchestrator"]["worktree_setup_timeout"] == 1200 + assert project.config["build"]["build_timeout"] == 240 + assert project.config["build"]["test_timeout"] == 360 + + +def test_adaptive_backoff_applies_to_next_wave(): + """An infra failure in wave 1 backs off concurrency for wave 2: the wave-2 + task sees a halved max_workers and a doubled timeout in the live config.""" + from unittest.mock import patch, MagicMock + import misterdev.agent as agent_mod + from misterdev.core.modes import BuildFlags + from misterdev.config import get_setting + + with tempfile.TemporaryDirectory() as td: + project = _budget_project(td) + project.config["orchestrator"].update( + { + "max_workers": 4, + "adaptive_infra_threshold": 0, # any infra fault triggers backoff + } + ) + # A fails on infra; B completes (wave 1). C depends on B (wave 2). + a = _mock_task("A") + b = _mock_task("B") + c = _mock_task("C", deps=["B"]) + + seen_workers = {} + + class _Exec: + def __init__(self, *a, **k): + pass + + def execute(self, t, proj): + seen_workers[t.id] = get_setting( + proj.config, "orchestrator", "max_workers" + ) + r = MagicMock() + if t.id == "A": + r.status = "failed" + r.logs = "Command timed out after 120s" + else: + r.status = "completed" + r.logs = "" + r.message = "" + return r + + def _run_command(self, *a, **k): + return True, "" + + with ( + patch.object(agent_mod, "MarkdownPlanExecutor", _Exec), + patch("misterdev.agent.Scratchpad"), + patch("misterdev.agent.RealTimeAligner"), + patch("misterdev.agent.ContractRegistry"), + patch("misterdev.agent.ChangeTracker"), + patch("misterdev.agent.StrategyOptimizer") as MockStrat, + ): + MockStrat.return_value.select_best_strategy.return_value = "iterative" + orch = agent_mod.ProjectOrchestrator() + orch._execute_tasks( + [a, b, c], project, BuildFlags(no_rollback=True), _fresh_report() + ) + + assert seen_workers["A"] == 4 # wave 1: full concurrency + assert seen_workers["C"] == 2 # wave 2: halved after the infra fault + # Config is restored to the configured base after the run. + assert get_setting(project.config, "orchestrator", "max_workers") == 4 + + +# --- run summary / failure taxonomy (P11) ----------------------------------- +def test_write_run_summary_writes_json_and_classifies(): + import json + from datetime import datetime, timezone + from unittest.mock import MagicMock + import misterdev.agent as agent_mod + from misterdev.core.models import Task, ExecutionResult + from misterdev.utils.file_utils import orchestrator_state_file + + with tempfile.TemporaryDirectory() as td: + td = Path(td) + report = _fresh_report() + report.start_time = datetime(2026, 1, 1, 0, 0, 0, tzinfo=timezone.utc) + report.end_time = datetime(2026, 1, 1, 0, 1, 30, tzinfo=timezone.utc) # 90s + report.completed_tasks = [ + Task(id="C1", description="d", project_ref="p"), + Task(id="C2", description="d", project_ref="p"), + ] + # A code failure (result logs) and a merge conflict (error stashed). + code_fail = Task(id="F1", description="d", project_ref="p") + code_fail.execution_history.append( + ExecutionResult(status="failed", message="x", logs="error TS2345: bad") + ) + merge_fail = Task(id="F2", description="d", project_ref="p") + merge_fail.processor_data["failure_text"] = "merge conflict: CONFLICT in env.ts" + report.failed_tasks = [code_fail, merge_fail] + # A parked task needing input. + parked = Task(id="D1", description="d", project_ref="p") + parked.execution_history.append( + ExecutionResult( + status="deferred", message="clarify the requirement", logs="" + ) + ) + report.deferred_tasks = [parked] + + project = MagicMock() + project.path = td + agent_mod.ProjectOrchestrator()._write_run_summary(project, report) + + data = json.loads(orchestrator_state_file(td, "run_summary.json").read_text()) + assert data["completed"] == 2 + assert data["failed"] == 2 + assert data["deferred"] == 1 + assert data["elapsed_seconds"] == 90.0 + assert data["failure_breakdown"] == { + "merge-conflict": 1, + "genuine-code-failure": 1, + "deferred-needs-input": 1, + } + + +# --- escalation ladder (P10) ------------------------------------------------ +def test_escalation_rung_reads_config_thresholds(): + from unittest.mock import MagicMock + from misterdev.task_executors.markdown_plan_executor import MarkdownPlanExecutor + + ex = MarkdownPlanExecutor() + project = MagicMock() + project.config = { + "orchestrator": { + "escalation_widen_after": 2, + "escalation_rewrite_after": 3, + "escalation_model_after": 4, + "escalation_decompose_after": 5, + } + } + assert ex._escalation_rung(project, 1) == "normal" + assert ex._escalation_rung(project, 2) == "widen_context" + assert ex._escalation_rung(project, 3) == "full_rewrite" + assert ex._escalation_rung(project, 4) == "stronger_model" + assert ex._escalation_rung(project, 5) == "decompose" + + +def test_decompose_substeps_from_acceptance_then_fallback(): + from misterdev.task_executors.markdown_plan_executor import MarkdownPlanExecutor + from misterdev.core.models import Task + + structured = Task( + id="T", + description="d", + project_ref="p", + acceptance_criteria="- endpoint returns 200\n- writes an audit row\n", + ) + steps = MarkdownPlanExecutor._decompose_substeps(structured) + assert len(steps) == 2 + assert all(s.startswith("Implement and verify:") for s in steps) + + # No structured criteria -> a generic isolate-then-extend split naming the task. + bare = Task( + id="T2", description="build the widget", project_ref="p", title="Widget" + ) + generic = MarkdownPlanExecutor._decompose_substeps(bare) + assert len(generic) == 2 + assert "Widget" in generic[0] + + +def test_escalation_spec_block_contains_verbatim_spec(): + from unittest.mock import MagicMock + from misterdev.task_executors.markdown_plan_executor import MarkdownPlanExecutor + from misterdev.core.models import Task + + t = Task( + id="T", + description="Add rate limiting to the API", + project_ref="p", + acceptance_criteria="429 after 100 req/min", + ) + block = MarkdownPlanExecutor()._escalation_spec_block( + MagicMock(), t, ["src/middleware.ts"] + ) + assert "Add rate limiting to the API" in block # objective verbatim + assert "429 after 100 req/min" in block # acceptance verbatim + assert "src/middleware.ts" in block # target file + + +def test_escalate_decompose_returns_deferred_with_named_substeps(): + from unittest.mock import MagicMock + from misterdev.task_executors.markdown_plan_executor import MarkdownPlanExecutor + from misterdev.core.models import Task + + t = Task( + id="T", + description="big task", + project_ref="p", + title="Big", + acceptance_criteria="- part a does X\n- part b does Y\n", + ) + # At depth >= 1 (a sub-step's own decompose) the base case still records the + # named sub-steps and defers rather than recursing (T4.1 recursion guard). + res = MarkdownPlanExecutor()._escalate_decompose( + MagicMock(), t, "last error", _depth=1 + ) + assert res.status == "deferred" + assert len(res.questions) >= 2 # named sub-steps surfaced as the decomposition + assert t.processor_data["_escalation_decomposed"] is True + assert t.processor_data["escalation_substeps"] == res.questions + + +def test_deferred_result_routed_to_deferred_not_failed(): + """A deferred task (walk-away input, or escalated to decomposition) lands in + report.deferred_tasks — NOT failed_tasks — and is not recorded as a terminal + progress failure, so a later run retries it.""" + from unittest.mock import patch, MagicMock + import misterdev.agent as agent_mod + from misterdev.core.modes import BuildFlags + + with tempfile.TemporaryDirectory() as td: + project = _budget_project(td) + task = _mock_task("T-def") + + class _Exec: + def __init__(self, *a, **k): + pass + + def execute(self, t, proj): + r = MagicMock() + r.status = "deferred" + return r + + def _run_command(self, *a, **k): + return True, "" + + with ( + patch.object(agent_mod, "MarkdownPlanExecutor", _Exec), + patch("misterdev.agent.Scratchpad"), + patch("misterdev.agent.RealTimeAligner"), + patch("misterdev.agent.ContractRegistry"), + patch("misterdev.agent.ChangeTracker"), + patch("misterdev.agent.StrategyOptimizer") as MS, + patch("misterdev.agent.ProgressTracker") as MP, + ): + MS.return_value.select_best_strategy.return_value = "iterative" + MP.return_value.completed = [] + MP.return_value.needs_rerun.return_value = True + orch = agent_mod.ProjectOrchestrator() + report = _fresh_report() + orch._execute_tasks([task], project, BuildFlags(no_rollback=True), report) + + assert task in report.deferred_tasks + assert task not in report.failed_tasks + MP.return_value.mark_failed.assert_not_called() # deferred != terminal fail + + def test_budget_exhausted_before_wave_defers_gracefully(): from unittest.mock import patch, MagicMock import misterdev.agent as agent_mod @@ -1776,6 +2718,78 @@ def test_executor_rejects_task_when_build_gate_stays_red(monkeypatch): assert not (repo / "feature.py").exists() +def test_keystone_decomposes_at_attempt_exhaustion(monkeypatch): + """A task that keeps failing real gates now DECOMPOSES (deferred, with named + sub-steps) at attempt exhaustion instead of dead-ending on a vague park — the + decompose rung is reachable even when escalation_decompose_after == the attempt + cap (default 3 == 3), which is the countless T007 keystone case.""" + from types import SimpleNamespace + from misterdev.task_executors.markdown_plan_executor import MarkdownPlanExecutor + + with tempfile.TemporaryDirectory() as td: + repo = Path(td) + _git(repo, "init") + _git(repo, "config", "user.email", "t@t.t") + _git(repo, "config", "user.name", "t") + (repo / "seed.py").write_text("X = 1\n") + _git(repo, "add", "-A") + _git(repo, "commit", "-m", "init") + + # DISTINCT applied edits (return 0,1,2,3,4) that all fail the build (!= 42): + # each is a real applied code failure that advances the ladder, unlike an + # identical edit that only applies once. + from misterdev.config import DEFAULT_CONFIG + from misterdev.core.execution.project import Project + from tests.test_llm_client import FakeLLMClient + from misterdev.llm.client import LLMResponse, LLMUsage + + monkeypatch.setenv("OPENROUTER_API_KEY", "test-key") + build_cmd = ( + 'python -c "import feature, sys; ' + 'sys.exit(0 if feature.answer() == 42 else 1)"' + ) + cfg = json.loads(json.dumps(DEFAULT_CONFIG)) + cfg["name"] = "fixture" + cfg["build_command"] = build_cmd + cfg.pop("test_command", None) + # Reflection would consume a FakeLLMClient response per retry and desync the + # distinct edits; off, each attempt's edit is the only LLM call. + cfg.setdefault("orchestrator", {})["reflection"] = False + project = Project(repo, cfg) + project.llm_client = FakeLLMClient( + responses=[ + LLMResponse( + content=f"```python:feature.py\ndef answer():\n return {i}\n```\n", + usage=LLMUsage(), + ) + for i in range(6) + ] + ) + task = SimpleNamespace( + id="T-keystone", + title="Hono app shell", + description="implement the app shell", + acceptance_criteria="- returns the error envelope\n- adds CORS\n", + files_to_modify=[], + files_to_create=["feature.py"], + context_files=[], + dependencies=[], + complexity="small", + category="feature", + processor_data={"strategy": "surgical"}, + execution_history=[], + ) + + result = MarkdownPlanExecutor().execute(task, project) + + # Reached the decompose rung at exhaustion instead of a plain fail/park. + assert result.status == "deferred" + assert task.processor_data.get("_escalation_decomposed") is True + assert len(task.processor_data.get("escalation_substeps", [])) >= 2 + # The named sub-steps are surfaced as the deferral questions. + assert result.questions == task.processor_data["escalation_substeps"] + + def test_completed_status_persists_to_committed_markdown(monkeypatch): """A completed task's status:completed is committed into its source .md and survives the merge — otherwise a finished devplan still reads 'pending' and a @@ -2405,7 +3419,7 @@ def _run_convergence_pipeline(gate_sequence, max_iterations, budget=100.0): flags = BuildFlags(budget=budget) orch = agent_mod.ProjectOrchestrator() - def fake_exec(tasks, project, flags, report): + def fake_exec(tasks, project, flags, report, progress_cb=None): counters["exec"] += 1 def fake_decompose(spec, assessment, mode, client, path, **kwargs): @@ -2468,7 +3482,7 @@ def _run_convergence_pipeline_with_cfg(gate_sequence, orchestrator_cfg): flags = BuildFlags(budget=100.0) orch = agent_mod.ProjectOrchestrator() - def fake_exec(tasks, project, flags, report): + def fake_exec(tasks, project, flags, report, progress_cb=None): counters["exec"] += 1 def fake_decompose(spec, assessment, mode, client, path, **kwargs): diff --git a/tests/test_parallel_worktree_prep.py b/tests/test_parallel_worktree_prep.py new file mode 100644 index 0000000..26a3554 --- /dev/null +++ b/tests/test_parallel_worktree_prep.py @@ -0,0 +1,71 @@ +"""M3 — a worktree is torn down if a prep step raises (no leak). + +`_prepare_task_worktree` creates the worktree, then primes/healthchecks it. If any +prep step raises, the wave cleanup (which only iterates fully-prepared tasks) would +never remove that worktree. The helper must tear it down and surface the error. +""" + +from types import SimpleNamespace + +from misterdev.core.execution.parallel import ParallelExecutionMixin + + +class _FakeGit: + def __init__(self): + self.removed = [] + self.deleted = [] + + def worktree_add(self, project, path, branch, new_branch=True): + return True, "" + + def worktree_remove(self, project, path): + self.removed.append(path) + + def branch_delete(self, project, branch): + self.deleted.append(branch) + + +def _task(): + return SimpleNamespace(id="T1") + + +def test_prep_raise_tears_down_worktree(tmp_path): + class _Ex(ParallelExecutionMixin): + def _prime_worktree_by_clone(self, *a, **k): + raise RuntimeError("clone blew up") + + git = _FakeGit() + prep, err = _Ex()._prepare_task_worktree( + None, git, _task(), tmp_path, True, None, None, 1, None + ) + assert prep is None + assert isinstance(err, RuntimeError) + assert git.removed and git.deleted # torn down, not leaked + + +def test_successful_prep_returns_tuple_no_teardown(tmp_path): + class _Ex(ParallelExecutionMixin): + def _prime_worktree_by_clone(self, *a, **k): + return True # primed; skips healthcheck + + git = _FakeGit() + prep, err = _Ex()._prepare_task_worktree( + None, git, _task(), tmp_path, True, None, None, 1, None + ) + assert err is None + task, wt_path, branch = prep + assert task.id == "T1" and branch.startswith("task/T1-") + assert not git.removed and not git.deleted + + +def test_worktree_add_failure_returns_error_no_teardown(tmp_path): + class _FailGit(_FakeGit): + def worktree_add(self, project, path, branch, new_branch=True): + return False, "add failed" + + git = _FailGit() + prep, err = ParallelExecutionMixin()._prepare_task_worktree( + None, git, _task(), tmp_path, True, None, None, 1, None + ) + assert prep is None and isinstance(err, RuntimeError) + assert not git.removed # nothing to tear down; it was never created diff --git a/tests/test_plan_store.py b/tests/test_plan_store.py new file mode 100644 index 0000000..6ad8f1f --- /dev/null +++ b/tests/test_plan_store.py @@ -0,0 +1,79 @@ +"""Persisted plan proposals with an approval gate.""" + +from dataclasses import dataclass + +from misterdev.core.planning import plan_store + + +@dataclass +class _Rec: + title: str + rationale: str + work_type: str + + +def test_save_assigns_ids_and_unapproved(tmp_path): + items = plan_store.save_plan( + tmp_path, + [ + _Rec("Add auth", "no login exists", "feature"), + _Rec("Fix flaky test", "CI is red", "debug"), + ], + ) + assert [it["id"] for it in items] == ["P-001", "P-002"] + assert all(it["approved"] is False for it in items) + assert items[0]["work_type"] == "feature" + # Persisted and reloadable. + assert plan_store.load_plan(tmp_path) == items + + +def test_save_accepts_dicts_and_skips_titleless(tmp_path): + items = plan_store.save_plan( + tmp_path, + [{"title": "Real"}, {"rationale": "no title"}], + ) + assert len(items) == 1 + assert items[0]["title"] == "Real" + assert items[0]["work_type"] == "complete" + + +def test_load_missing_returns_none(tmp_path): + assert plan_store.load_plan(tmp_path) is None + + +def test_approve_all(tmp_path): + plan_store.save_plan(tmp_path, [_Rec("a", "", "feature"), _Rec("b", "", "debug")]) + updated = plan_store.set_approval(tmp_path, approve_all=True) + assert all(it["approved"] for it in updated) + assert len(plan_store.approved_items(tmp_path)) == 2 + + +def test_approve_and_reject_subset(tmp_path): + plan_store.save_plan( + tmp_path, + [_Rec("a", "", "feature"), _Rec("b", "", "debug"), _Rec("c", "", "feature")], + ) + plan_store.set_approval(tmp_path, item_ids=["P-001", "P-003"]) + assert {it["id"] for it in plan_store.approved_items(tmp_path)} == { + "P-001", + "P-003", + } + # Rejecting one flips it back. + plan_store.set_approval(tmp_path, reject_ids=["P-001"]) + assert {it["id"] for it in plan_store.approved_items(tmp_path)} == {"P-003"} + + +def test_reject_wins_a_tie(tmp_path): + plan_store.save_plan(tmp_path, [_Rec("a", "", "feature")]) + updated = plan_store.set_approval( + tmp_path, item_ids=["P-001"], reject_ids=["P-001"] + ) + assert updated[0]["approved"] is False + + +def test_set_approval_without_plan_returns_none(tmp_path): + assert plan_store.set_approval(tmp_path, approve_all=True) is None + + +def test_approved_items_empty_without_plan(tmp_path): + assert plan_store.approved_items(tmp_path) == [] diff --git a/tests/test_proactive_keystone_split.py b/tests/test_proactive_keystone_split.py new file mode 100644 index 0000000..8e39b49 --- /dev/null +++ b/tests/test_proactive_keystone_split.py @@ -0,0 +1,60 @@ +"""T4.3 — a keystone (high-fan-in) task is split into chained sub-units. + +Splitting a task many others depend on reduces per-attempt blast radius. The split +partitions its files across chained sub-units and rewires every dependent to the FINAL +sub-unit, so the dependency ordering is preserved. Non-keystones and keystones with too +few files to partition are unchanged. +""" + +from misterdev.core.models import Task +from misterdev.core.planning.decomposer import split_keystone_tasks + + +def _task(tid, deps=None, create=None, modify=None): + return Task( + id=tid, + description="x", + project_ref=".", + dependencies=list(deps or []), + files_to_create=list(create or []), + files_to_modify=list(modify or []), + acceptance_criteria="pytest passes", + ) + + +def _keystone_plan(n_deps=3, files=("a.py", "b.py")): + key = _task("K", modify=list(files)) + deps = [_task(f"D{i}", deps=["K"]) for i in range(n_deps)] + return [key] + deps + + +def test_keystone_is_split_and_dependents_rewired(): + tasks = _keystone_plan(n_deps=3, files=("a.py", "b.py")) + out = split_keystone_tasks(tasks, fanin_threshold=3, min_units=2) + ids = {t.id for t in out} + assert "K" not in ids # replaced by sub-units + subs = sorted(t.id for t in out if t.id.startswith("K-part")) + assert len(subs) == 2 + # Later sub-unit chains on the earlier one. + part2 = next(t for t in out if t.id == "K-part2") + assert "K-part1" in part2.dependencies + # Every original dependent now waits on the FINAL sub-unit, not "K". + final = subs[-1] + for d in [t for t in out if t.id.startswith("D")]: + assert final in d.dependencies and "K" not in d.dependencies + # Files were partitioned across the units (no loss, no duplication). + part1 = next(t for t in out if t.id == "K-part1") + assert set(part1.files_to_modify) | set(part2.files_to_modify) == {"a.py", "b.py"} + assert not (set(part1.files_to_modify) & set(part2.files_to_modify)) + + +def test_low_fanin_task_unchanged(): + tasks = _keystone_plan(n_deps=1, files=("a.py", "b.py")) + out = split_keystone_tasks(tasks, fanin_threshold=3) + assert {t.id for t in out} == {"K", "D0"} + + +def test_keystone_with_one_file_not_split(): + tasks = _keystone_plan(n_deps=5, files=("only.py",)) + out = split_keystone_tasks(tasks, fanin_threshold=3, min_units=2) + assert "K" in {t.id for t in out} diff --git a/tests/test_progress.py b/tests/test_progress.py index 3557770..1ab84bb 100644 --- a/tests/test_progress.py +++ b/tests/test_progress.py @@ -1,7 +1,9 @@ +import json import tempfile from pathlib import Path from misterdev.core.execution.progress import ProgressTracker +from misterdev.utils.file_utils import orchestrator_state_file def test_mark_and_check(): @@ -37,6 +39,55 @@ def test_mark_completed_clears_failed(): assert pt.is_done("T-001") +def test_completed_after_failed_is_only_completed(): + """Marking completed after failed leaves the task in exactly one terminal + state: completed wins and the failed entry is cleared.""" + with tempfile.TemporaryDirectory() as td: + pt = ProgressTracker(Path(td)) + pt.mark_failed("T-001") + pt.mark_completed("T-001") + assert "T-001" in pt.completed + assert "T-001" not in pt.failed + + +def test_failed_after_completed_is_rejected(): + """The reverse is rejected: a task already completed is never re-listed as + failed, so it stays only in completed (single terminal state).""" + with tempfile.TemporaryDirectory() as td: + pt = ProgressTracker(Path(td)) + pt.mark_completed("T-001") + pt.mark_failed("T-001") + assert "T-001" in pt.completed + assert "T-001" not in pt.failed + # And it survives a reload — the rejection was persisted, not just in-memory. + assert "T-001" not in ProgressTracker(Path(td)).failed + + +def test_load_reconciles_poisoned_ledger(): + """An existing progress.json listing a task in BOTH completed and failed (the + observed T002/T062a poisoning) self-heals on load: it is dropped from failed + and the healed state is written back to disk.""" + with tempfile.TemporaryDirectory() as td: + state = orchestrator_state_file(Path(td), "progress.json") + state.parent.mkdir(parents=True, exist_ok=True) + state.write_text( + json.dumps( + { + "completed": ["T002", "T062a", "T003"], + "failed": ["T002", "T062a", "T099"], + "hashes": {}, + } + ), + encoding="utf-8", + ) + pt = ProgressTracker(Path(td)) + assert pt.completed == {"T002", "T062a", "T003"} + assert pt.failed == {"T099"} # poisoned entries dropped, real failure kept + # Healed state was persisted, so a fresh load sees the clean ledger. + on_disk = json.loads(state.read_text(encoding="utf-8")) + assert set(on_disk["failed"]) == {"T099"} + + def test_reset(): with tempfile.TemporaryDirectory() as td: pt = ProgressTracker(Path(td)) diff --git a/tests/test_reference_digest.py b/tests/test_reference_digest.py new file mode 100644 index 0000000..e60b79d --- /dev/null +++ b/tests/test_reference_digest.py @@ -0,0 +1,88 @@ +"""Reference-implementation digest: read-only structural map for porting.""" + +import tempfile +from pathlib import Path + +import pytest + +from misterdev.analyzers.reference_digest import build_reference_digest + + +def _ref_tree() -> Path: + """A small multi-file reference tree under a fresh temp dir.""" + root = Path(tempfile.mkdtemp(prefix="ref-src-")) + (root / "core").mkdir() + (root / "core" / "engine.py").write_text( + "class Engine:\n" + " def start(self):\n return 1\n\n" + "def boot():\n return Engine()\n" + ) + (root / "util.py").write_text("def helper(x):\n return x + 1\n") + return root + + +def test_digest_names_reference_and_lists_symbols(): + ref = _ref_tree() + cache = Path(tempfile.mkdtemp(prefix="cache-")) + digest = build_reference_digest(ref, cache_dir=cache) + + assert ref.name in digest + assert "Reference implementation to port from" in digest + # Symbols from both files/dirs appear in the map. + assert "Engine" in digest + assert "boot" in digest + assert "helper" in digest + assert "core/engine.py" in digest + + +def test_digest_never_writes_into_reference_tree(): + ref = _ref_tree() + before = {p.relative_to(ref) for p in ref.rglob("*")} + cache = Path(tempfile.mkdtemp(prefix="cache-")) + + build_reference_digest(ref, cache_dir=cache) + + after = {p.relative_to(ref) for p in ref.rglob("*")} + assert before == after, "reference tree must not be mutated" + assert not (ref / ".orchestrator").exists() + # The cache landed in the redirected location instead. + assert (cache / "reference_topography_cache.json").exists() + + +def test_digest_default_cache_dir_stays_off_reference_tree(): + ref = _ref_tree() + # No cache_dir: a throwaway temp dir is used, never the reference. + build_reference_digest(ref) + assert not (ref / ".orchestrator").exists() + + +def test_digest_missing_path_raises(): + with pytest.raises(ValueError): + build_reference_digest("/no/such/reference/dir/xyz") + + +def test_digest_file_path_raises(): + ref = _ref_tree() + with pytest.raises(ValueError): + build_reference_digest(ref / "util.py") + + +def test_digest_rejects_nonpositive_max_chars(): + ref = _ref_tree() + with pytest.raises(ValueError): + build_reference_digest(ref, max_chars=0) + + +def test_digest_empty_tree_returns_header_with_note(): + empty = Path(tempfile.mkdtemp(prefix="ref-empty-")) + digest = build_reference_digest(empty) + assert "Reference implementation to port from" in digest + assert "no source symbols" in digest + + +def test_digest_truncates_large_map_with_note(): + ref = _ref_tree() + digest = build_reference_digest(ref, max_chars=40) + assert "truncated" in digest + # Header is always present; only the map body is bounded. + assert "Reference implementation to port from" in digest diff --git a/tests/test_requirements.py b/tests/test_requirements.py index f2d444a..0af712e 100644 --- a/tests/test_requirements.py +++ b/tests/test_requirements.py @@ -84,6 +84,27 @@ def test_gating_only_foundational_missing_accounts(): ] gating = R.gating_requirements(reqs, tasks, threshold=3) assert [g["key"] for g in gating] == ["CLOUDFLARE_ACCOUNT"] # env is advisory + + +def test_gating_skips_answered_accounts(): + # A foundational account need the user has answered (a typed decision, e.g. "skip + # the deploy") no longer stops the run — the user made the call. + tasks = [ + _t("T1", "deploy to cloudflare"), + _t("T2", deps=["T1"]), + _t("T3", deps=["T2"]), + _t("T4", deps=["T1"]), + ] + reqs = [ + { + "key": "CLOUDFLARE_ACCOUNT", + "kind": "account", + "task_ids": ["T1"], + "satisfied": False, + "answered": True, + }, + ] + assert R.gating_requirements(reqs, tasks, threshold=3) == [] # Same account need but leaf (no dependents) does not gate. leaf = [_t("L1", "deploy to cloudflare")] assert ( @@ -143,11 +164,38 @@ def test_requirements_book_writes_and_reads_decisions(tmp_path: Path): ) assert book.md_path.exists() assert book.load_answers() == {} - txt = book.md_path.read_text(encoding="utf-8").replace( - "_(provide this, or write your decision here)_", "use D1", 1 - ) + # Every unsatisfied requirement is answerable, not just decisions: the account + # and the decision each carry an Answer line to fill. + assert book.md_path.read_text(encoding="utf-8").count("- Answer:") == 2 + # Fill the account requirement (the first block) and the decision one. + txt = book.md_path.read_text(encoding="utf-8") + txt = txt.replace( + "_(provide this, or write your decision here)_", "skip the deploy", 1 + ).replace("_(provide this, or write your decision here)_", "use D1", 1) book.md_path.write_text(txt, encoding="utf-8") - assert book.load_answers().get("DB_CHOICE") == "use D1" + answers = book.load_answers() + assert answers.get("CLOUDFLARE_ACCOUNT") == "skip the deploy" + assert answers.get("DB_CHOICE") == "use D1" + # A re-write preserves both typed answers. + book.write( + [ + { + "key": "CLOUDFLARE_ACCOUNT", + "kind": "account", + "summary": "cf", + "task_ids": ["T1"], + "satisfied": False, + }, + { + "key": "DB_CHOICE", + "kind": "decision", + "summary": "which db", + "task_ids": ["T2"], + "satisfied": False, + }, + ] + ) + assert book.load_answers().get("CLOUDFLARE_ACCOUNT") == "skip the deploy" # --- run_project smart gate: stop vs proceed -------------------------------- diff --git a/tests/test_run_summary.py b/tests/test_run_summary.py new file mode 100644 index 0000000..9a0d032 --- /dev/null +++ b/tests/test_run_summary.py @@ -0,0 +1,117 @@ +"""Per-run failure taxonomy (core.execution): classifier routing + aggregation. + +Distinct from core.learning.failure_taxonomy (the cognitive-cause classifier); +this is the operational end-of-run breakdown that feeds run_summary.json. +""" + +from misterdev.core.execution.failure_taxonomy import ( + CATEGORIES, + build_run_summary, + classify_failure, +) + + +def test_classify_each_category(): + assert classify_failure("failed", "Command timed out after 120s") == "infra" + assert ( + classify_failure("failed", "please run `wrangler login`, not logged in") + == "blocked-external" + ) + assert ( + classify_failure("failed", "merge conflict: CONFLICT (content) in env.ts") + == "merge-conflict" + ) + assert ( + classify_failure("failed", "### Acceptance criterion not met\n...") + == "acceptance-unmet" + ) + assert ( + classify_failure("failed", "error TS2345: bad type") == "genuine-code-failure" + ) + assert ( + classify_failure("deferred", "how should I proceed with this?") + == "deferred-needs-input" + ) + + +def test_signal_wins_over_status(): + """A blocked/infra signal is labelled as such regardless of status — that IS + why the task parked or failed.""" + assert ( + classify_failure("deferred", "a required API key is missing") + == "blocked-external" + ) + assert classify_failure("failed", "ENOSPC: no space left on device") == "infra" + + +def test_deferred_without_signal_is_needs_input(): + assert ( + classify_failure("deferred", "please clarify the requirement") + == "deferred-needs-input" + ) + + +def test_every_category_reachable(): + produced = { + classify_failure("failed", "Command timed out after 120s"), + classify_failure("failed", "wrangler login required"), + classify_failure("failed", "merge conflict in a.ts"), + classify_failure("failed", "Acceptance criteria not met"), + classify_failure("failed", "AssertionError: 1 != 2"), + classify_failure("deferred", "clarify please"), + } + assert produced == set(CATEGORIES) + + +def test_build_run_summary_counts_and_breakdown(): + summary = build_run_summary( + completed=5, + failed_items=[ + ("T1", "Command timed out after 120s"), + ("T2", "error TS2345: bad type"), + ("T3", "another AssertionError: x"), + ], + deferred_items=[("T4", "a required API key is missing")], + elapsed_seconds=93.44, + ) + assert summary["completed"] == 5 + assert summary["failed"] == 3 + assert summary["deferred"] == 1 + assert summary["elapsed_seconds"] == 93.4 # rounded to one decimal + assert summary["failure_breakdown"] == { + "infra": 1, + "blocked-external": 1, + "genuine-code-failure": 2, + } + assert summary["top_obstacle"] == "genuine-code-failure" # the 2 outweigh the 1s + assert "genuine-code-failure" in summary["exemplars"] + assert summary["exemplars"]["infra"] == "Command timed out after 120s" + + +def test_top_obstacle_tie_breaks_to_more_specific(): + """One of each category → the tie breaks toward the earliest (most specific) + category, infra.""" + summary = build_run_summary( + completed=0, + failed_items=[ + ("T1", "Command timed out after 120s"), + ("T2", "error TS2345"), + ], + deferred_items=[("T3", "clarify please")], + elapsed_seconds=1.0, + ) + assert summary["top_obstacle"] == "infra" + + +def test_empty_run_has_no_breakdown_or_obstacle(): + summary = build_run_summary(3, [], [], 10.0) + assert summary["failure_breakdown"] == {} + assert summary["top_obstacle"] is None + assert summary["exemplars"] == {} + + +def test_exemplar_skips_fences_and_headers(): + summary = build_run_summary( + 0, [("T1", "```\n### header\nreal error line here\n")], [], 1.0 + ) + assert summary["exemplars"]["genuine-code-failure"] == "real error line here" diff --git a/tests/test_sovereign.py b/tests/test_sovereign.py index f67fd80..372e1e6 100644 --- a/tests/test_sovereign.py +++ b/tests/test_sovereign.py @@ -170,3 +170,43 @@ def test_extract_code_block_no_fence(): def test_extract_code_block_multiple(): response = "First:\n```\nblock1\n```\nSecond:\n```\nblock2\n```" assert _extract_code_block(response) == "block1" + + +def test_load_tolerates_valid_json_of_wrong_shape(tmp_path): + import json + + from misterdev.core.planning.sovereign import RealTimeAligner + + d = tmp_path / ".orchestrator" + d.mkdir(parents=True) + (d / "consensus.json").write_text(json.dumps([1, 2, 3])) # valid JSON, wrong shape + a = RealTimeAligner(tmp_path) + assert a.get_consensus_context() == "No prior decisions recorded." + a.certify_decision("use X", "because") + assert "use X" in a.get_consensus_context() + + +def test_load_missing_keys_default(tmp_path): + import json + + from misterdev.core.planning.sovereign import RealTimeAligner + + d = tmp_path / ".orchestrator" + d.mkdir(parents=True) + (d / "consensus.json").write_text(json.dumps({"unrelated": 1})) + a = RealTimeAligner(tmp_path) + a.certify_decision("d", "r") + assert "d" in a.get_consensus_context() + + +def test_load_preserves_valid_decisions(tmp_path): + import json + + from misterdev.core.planning.sovereign import RealTimeAligner + + d = tmp_path / ".orchestrator" + d.mkdir(parents=True) + (d / "consensus.json").write_text( + json.dumps({"invariants": [], "decisions": [{"decision": "keep", "rationale": "r"}]}) + ) + assert "keep" in RealTimeAligner(tmp_path).get_consensus_context() diff --git a/tests/test_spec_as_tests_default.py b/tests/test_spec_as_tests_default.py new file mode 100644 index 0000000..e501c7d --- /dev/null +++ b/tests/test_spec_as_tests_default.py @@ -0,0 +1,71 @@ +"""T1.1 — spec-as-tests is default-on, with the baseline-safety invariant locked. + +Reproduction-first is now the default (`spec_as_tests=True`), not opt-in. The +"hardened so the baseline does not go red inside a wave" property already exists — +the generated test is written under `.orchestrator/spec_tests/`, OUTSIDE the project +suite, so it can never be collected by the project's own test run and therefore can +never flip the integration-gate baseline. This locks that invariant so default-on is +safe. (Compiled-language spec tests are deliberately still generated as injected +targets even when unrunnable as a gate — that behavior is covered by +test_spec_tests.py and intentionally unchanged.) +""" + +from pathlib import Path +from types import SimpleNamespace + +from misterdev.config import DEFAULT_CONFIG +from misterdev.task_executors.markdown_plan_executor.critic_spec_mixin import ( + CriticSpecMixin, +) + + +class _Ex(CriticSpecMixin): + pass + + +class _Client: + def __init__(self): + self.calls = 0 + + def generate_code(self, prompt, system): + self.calls += 1 + return "```python\nassert False\n```" + + +def _project(tmp_path): + return SimpleNamespace( + path=tmp_path, + llm_client=_Client(), + config={ + "orchestrator": {"spec_as_tests": True}, + "language": "python", + "test_command": "pytest", + }, + ) + + +def _task(): + return SimpleNamespace( + id="T1", acceptance_criteria="The widget must render.", description="render it" + ) + + +def test_spec_as_tests_defaults_on(): + assert DEFAULT_CONFIG["orchestrator"]["spec_as_tests"] is True + + +def test_generated_spec_written_outside_project_suite(tmp_path): + # Hardening invariant: default-on is safe because the generated test lives in + # the baseline-safe .orchestrator/ lane, never the collected project suite. + project = _project(tmp_path) + path, source = _Ex()._maybe_generate_spec_test(project, _task()) + assert path is not None and source + assert Path(path).parent == tmp_path / ".orchestrator" / "spec_tests" + + +def test_default_on_generation_actually_runs(tmp_path): + # With the default flipped on, a supported-language task generates (control). + project = _project(tmp_path) + path, _ = _Ex()._maybe_generate_spec_test(project, _task()) + assert path is not None + assert project.llm_client.calls == 1 diff --git a/tests/test_spec_tests.py b/tests/test_spec_tests.py index 58d7a8e..c9e8d4d 100644 --- a/tests/test_spec_tests.py +++ b/tests/test_spec_tests.py @@ -143,16 +143,18 @@ def test_write_spec_test_creates_dir_and_file(tmp_path): # --- wiring into the executor (per-task) ------------------------------------ -def test_spec_as_tests_flags_readable_and_off_by_default(): +def test_spec_as_tests_flags_readable_and_on_by_default(): from misterdev.config import get_setting, DEFAULT_CONFIG - assert DEFAULT_CONFIG["orchestrator"]["spec_as_tests"] is False + # Default-on (reproduction-first is the default), but still ADVISORY: a red + # spec test does not block acceptance unless spec_as_tests_block is set. + assert DEFAULT_CONFIG["orchestrator"]["spec_as_tests"] is True assert DEFAULT_CONFIG["orchestrator"]["spec_as_tests_block"] is False assert ( get_setting( - {"orchestrator": {"spec_as_tests": True}}, "orchestrator", "spec_as_tests" + {"orchestrator": {"spec_as_tests": False}}, "orchestrator", "spec_as_tests" ) - is True + is False ) @@ -202,7 +204,9 @@ class _Proj: assert ".orchestrator" in p.parts and "spec_tests" in p.parts -def test_maybe_generate_off_by_default(tmp_path): +def test_maybe_generate_on_by_default(tmp_path): + # Default-on: an empty config resolves spec_as_tests to the default (True) via + # DEFAULT_CONFIG, so generation proceeds without any explicit opt-in. from misterdev.task_executors.markdown_plan_executor import ( MarkdownPlanExecutor, ) @@ -212,6 +216,22 @@ class _Proj: config = {} llm_client = _SpecClient() + path, source = MarkdownPlanExecutor()._maybe_generate_spec_test( + _Proj(), _task(tid="t1") + ) + assert path is not None and source + + +def test_maybe_generate_explicit_off_disables(tmp_path): + from misterdev.task_executors.markdown_plan_executor import ( + MarkdownPlanExecutor, + ) + + class _Proj: + path = tmp_path + config = {"orchestrator": {"spec_as_tests": False}} + llm_client = _SpecClient() + assert MarkdownPlanExecutor()._maybe_generate_spec_test( _Proj(), _task(tid="t1") ) == (None, None) diff --git a/tests/test_stall_fresh_candidate_reset.py b/tests/test_stall_fresh_candidate_reset.py new file mode 100644 index 0000000..d76661e --- /dev/null +++ b/tests/test_stall_fresh_candidate_reset.py @@ -0,0 +1,50 @@ +"""T3.2 — on a detected stall, reset to a fresh candidate (clean task base). + +`_reset_to_task_base` returns the working tree to the task's clean base so the next +attempt re-derives from scratch, but — unlike `_abort_task` — stays ON the task branch +and in the retry loop (it must NOT check out the base or delete the branch). This drives +the helper directly with a duck fixture. +""" + +from types import SimpleNamespace + +from misterdev.task_executors.markdown_plan_executor.git_mixin import GitMixin + + +class _Ex(GitMixin): + def __init__(self): + self.git_cmds = [] + self.reverted = None + self.orphans_cleaned = False + + def _git(self, project, cmd): + self.git_cmds.append(cmd) + return (True, "") + + def _revert_files(self, project, snapshot): + self.reverted = snapshot + + def _clean_task_orphans(self, project, untracked_before): + self.orphans_cleaned = True + + +def _project(): + return SimpleNamespace(topography=None) + + +def test_git_mode_resets_hard_without_leaving_branch(): + ex = _Ex() + ex._reset_to_task_base(_project(), "task-branch", "main", None, set()) + assert any("reset --hard" in c for c in ex.git_cmds) + # Must NOT end the task: no checkout of base, no branch delete. + assert not any("checkout" in c for c in ex.git_cmds) + assert not any("branch -D" in c for c in ex.git_cmds) + assert ex.orphans_cleaned + + +def test_snapshot_mode_reverts_files(): + ex = _Ex() + snap = {"a.py": "original"} + ex._reset_to_task_base(_project(), None, None, snap, set()) + assert ex.reverted == snap + assert ex.orphans_cleaned diff --git a/tests/test_symbol_graph_file_index.py b/tests/test_symbol_graph_file_index.py new file mode 100644 index 0000000..b62b059 --- /dev/null +++ b/tests/test_symbol_graph_file_index.py @@ -0,0 +1,58 @@ +"""H4 — SymbolGraph's file-scoped queries use a memoized per-file index. + +file_symbols / _match_files / symbol_at_line were each a full O(all-symbols) scan. +A memoized `(file_path -> [(key, node)])` index makes them file-local and is rebuilt +only when the symbol set is replaced/resized. Behavior is unchanged; this locks it. +""" + +from misterdev.core.context.topography import SymbolGraph, SymbolNode + + +def _graph(*nodes): + g = SymbolGraph.__new__(SymbolGraph) + g.symbols = {f"{n.file_path}:{n.name}": n for n in nodes} + return g + + +def test_file_index_groups_by_file_sorted_by_start_line(): + g = _graph( + SymbolNode("b", "a.py", "function", 20, 25, "x"), + SymbolNode("a", "a.py", "function", 5, 10, "x"), + SymbolNode("z", "other.py", "function", 0, 3, "x"), + ) + idx = g._file_index() + assert set(idx) == {"a.py", "other.py"} + assert [node.name for _k, node in idx["a.py"]] == ["a", "b"] # sorted by start_line + + +def test_file_index_is_memoized_and_rebuilds_on_replacement(): + g = _graph(SymbolNode("a", "a.py", "function", 0, 5, "x")) + first = g._file_index() + assert g._file_index() is first # same object while symbols unchanged + g.symbols = {"b.py:b": SymbolNode("b", "b.py", "function", 0, 5, "x")} + assert g._file_index() is not first # rebuilt after the symbol set is replaced + assert set(g._file_index()) == {"b.py"} + + +def test_public_methods_unchanged(): + g = _graph( + SymbolNode("Cls", "a.py", "class", 0, 100, "x"), + SymbolNode("m", "a.py", "method", 40, 50, "x"), + ) + assert [s.name for s in g.file_symbols("a.py")] == ["Cls", "m"] + # narrowest enclosing span wins (method over class) + assert g.symbol_at_line("a.py", 45) == "a.py:m" + assert g.symbol_at_line("a.py", 5) == "a.py:Cls" + assert g.symbol_at_line("a.py", 200) is None + assert g._match_files("a.py") == {"a.py"} + + +def test_unique_suffix_match_preserved(): + g = _graph(SymbolNode("h", "frontend/src/app.ts", "function", 5, 15, "x")) + assert g._match_files("src/app.ts") == {"frontend/src/app.ts"} + # ambiguous suffix -> no match + g2 = _graph( + SymbolNode("f", "a/x.py", "function", 5, 15, "x"), + SymbolNode("g", "b/x.py", "function", 5, 15, "x"), + ) + assert g2._match_files("x.py") == set() diff --git a/tests/test_symbol_lookup_on_diagnostic.py b/tests/test_symbol_lookup_on_diagnostic.py new file mode 100644 index 0000000..a70a319 --- /dev/null +++ b/tests/test_symbol_lookup_on_diagnostic.py @@ -0,0 +1,79 @@ +"""T2.2 — a diagnostic that NAMES a symbol surfaces that symbol's definition. + +On "cannot find X" / "expected `A`, found `B`" / tsc "Cannot find name 'X'", the +resolver must look X up in the symbol graph and surface X's DEFINITION (where it is +declared) into the repair context — not merely attribute the error line to its +enclosing symbol. Only names that resolve to a real project symbol are surfaced, so +compiler primitives (`u32`, `string`) never add noise. +""" + +from pathlib import Path + +from misterdev.core.context.topography import SymbolGraph, SymbolNode +from misterdev.core.execution.error_resolver import ErrorResolver + + +def _graph(*nodes): + g = SymbolGraph.__new__(SymbolGraph) + g.symbols = {f"{n.file_path}:{n.name}": n for n in nodes} + return g + + +def test_rust_cannot_find_function_surfaces_definition(): + graph = _graph( + SymbolNode("helper", "src/util.rs", "function", 11, 15, "fn helper() {}") + ) + r = ErrorResolver(Path("."), graph) + out = r.format_for_llm( + [], error_output="error[E0425]: cannot find function `helper` in this scope" + ) + assert "Referenced symbol definitions" in out + assert "`helper`" in out + assert "src/util.rs:12" in out # 0-indexed row 11 -> 1-indexed line 12 + assert "fn helper" in out # the definition body is included + + +def test_tsc_cannot_find_name_surfaces_definition(): + graph = _graph( + SymbolNode("Widget", "src/widget.ts", "class", 4, 30, "class Widget {}") + ) + r = ErrorResolver(Path("."), graph) + out = r.format_for_llm( + [], error_output="src/a.ts(3,5): error TS2304: Cannot find name 'Widget'." + ) + assert "`Widget`" in out + assert "src/widget.ts:5" in out + + +def test_type_mismatch_surfaces_named_type_definition(): + graph = _graph( + SymbolNode("Celsius", "src/units.rs", "struct", 9, 12, "struct Celsius(f64);") + ) + r = ErrorResolver(Path("."), graph) + out = r.format_for_llm([], error_output="expected `Celsius`, found `Fahrenheit`") + assert "`Celsius`" in out + assert "src/units.rs:10" in out + + +def test_unknown_or_primitive_name_is_not_surfaced(): + graph = _graph(SymbolNode("helper", "src/util.rs", "function", 11, 15, "x")) + r = ErrorResolver(Path("."), graph) + # `u32` is not a project symbol -> no referenced-definition section at all. + out = r.format_for_llm([], error_output="expected `u32`, found `&str`") + assert "Referenced symbol definitions" not in out + + +def test_no_graph_no_referenced_section(): + r = ErrorResolver(Path(".")) # no graph + out = r.format_for_llm( + [], error_output="cannot find function `helper` in this scope" + ) + assert "Referenced symbol definitions" not in out + + +def test_error_output_omitted_preserves_prior_behavior(): + # Backward compatibility: calling without error_output yields only attribution. + graph = _graph(SymbolNode("helper", "src/util.rs", "function", 11, 15, "x")) + r = ErrorResolver(Path("."), graph) + out = r.format_for_llm([]) + assert out == "" diff --git a/tests/test_task_size_verifiability_invariant.py b/tests/test_task_size_verifiability_invariant.py new file mode 100644 index 0000000..75c9c4e --- /dev/null +++ b/tests/test_task_size_verifiability_invariant.py @@ -0,0 +1,52 @@ +"""T4.2 — decomposed tasks are checked against a size/verifiability invariant. + +`enforce_task_invariants` flags a task that touches too many files (size) or carries +a blank/placeholder acceptance criterion (verifiability), recording the reason on +processor_data so downstream can split or reject it. A well-formed task is untouched. +""" + +from misterdev.core.models import Task +from misterdev.core.planning.decomposer import enforce_task_invariants + + +def _task(tid, files=None, acceptance="pytest tests/test_x.py passes"): + return Task( + id=tid, + description="do a thing", + project_ref=".", + files_to_modify=list(files or ["a.py"]), + acceptance_criteria=acceptance, + ) + + +def _violations(task): + return (task.processor_data or {}).get("invariant_violations") or [] + + +def test_wellformed_task_has_no_violations(): + t = _task("T1") + enforce_task_invariants([t]) + assert _violations(t) == [] + + +def test_oversized_task_flagged(): + t = _task("T2", files=[f"f{i}.py" for i in range(25)]) + enforce_task_invariants([t], max_files=20) + assert any("file" in v.lower() for v in _violations(t)) + + +def test_blank_acceptance_flagged(): + t = _task("T3", acceptance="") + enforce_task_invariants([t]) + assert any("accept" in v.lower() or "verif" in v.lower() for v in _violations(t)) + + +def test_placeholder_acceptance_flagged(): + t = _task("T4", acceptance="works") + enforce_task_invariants([t]) + assert _violations(t) + + +def test_returns_the_task_list(): + tasks = [_task("T5")] + assert enforce_task_invariants(tasks) is tasks diff --git a/tests/test_validator.py b/tests/test_validator.py index 790f7eb..733d06c 100644 --- a/tests/test_validator.py +++ b/tests/test_validator.py @@ -308,3 +308,25 @@ def fake_run_cmd(cmd, cwd, env_activate=None, timeout=180): ) assert calls["lint"] == 240 # explicit lint_timeout, not test_timeout assert calls["t"] == 300 + + +def test_parse_test_counts_sums_multiple_dotnet_projects(): + # A multi-project VSTest solution prints one summary block per project; the gate + # must sum them, not read only the first (an undercount can hide a regression). + out = ( + "Failed: 1, Passed: 5, Skipped: 0, Total: 6\n" + "Failed: 3, Passed: 2, Skipped: 0, Total: 5\n" + ) + assert _parse_test_counts(out) == (11, 4) + + +def test_parse_test_counts_sums_multiple_dotnet_alt_projects(): + out = ( + "Total tests: 6. Passed: 5. Failed: 1.\n" + "Total tests: 5. Passed: 2. Failed: 3.\n" + ) + assert _parse_test_counts(out) == (11, 4) + + +def test_parse_test_counts_single_dotnet_project_unchanged(): + assert _parse_test_counts("Failed: 2, Passed: 3, Skipped: 0, Total: 5") == (5, 2) diff --git a/tests/test_venv_env.py b/tests/test_venv_env.py new file mode 100644 index 0000000..df77e50 --- /dev/null +++ b/tests/test_venv_env.py @@ -0,0 +1,43 @@ +"""M2 — venv setup subprocess is timeout-bounded and captures output. + +A hung `pip install` / `python -m venv` with no timeout blocks the whole run +indefinitely. setup() must bound each command with a timeout and fail cleanly +(returning False) on a timeout instead of hanging. +""" + +import subprocess + +import misterdev.environments.venv_env as ve +from misterdev.environments.venv_env import VenvEnvironmentManager + + +def _mgr(tmp_path): + return VenvEnvironmentManager({"root_dir": "vv"}, tmp_path) + + +def test_setup_passes_a_timeout(tmp_path, monkeypatch): + seen = {} + + def fake_run(cmd, **kw): + seen.update(kw) + return subprocess.CompletedProcess(cmd, 0, "", "") + + monkeypatch.setattr(ve.subprocess, "run", fake_run) + assert _mgr(tmp_path).setup() is True + assert "timeout" in seen and seen["timeout"] > 0 + + +def test_setup_returns_false_on_timeout(tmp_path, monkeypatch): + def fake_run(cmd, **kw): + raise subprocess.TimeoutExpired(cmd, kw.get("timeout", 1)) + + monkeypatch.setattr(ve.subprocess, "run", fake_run) + assert _mgr(tmp_path).setup() is False + + +def test_setup_returns_false_on_command_error(tmp_path, monkeypatch): + def fake_run(cmd, **kw): + raise subprocess.CalledProcessError(1, cmd, output="", stderr="boom") + + monkeypatch.setattr(ve.subprocess, "run", fake_run) + assert _mgr(tmp_path).setup() is False diff --git a/tests/test_wave_partition.py b/tests/test_wave_partition.py new file mode 100644 index 0000000..a9ac787 --- /dev/null +++ b/tests/test_wave_partition.py @@ -0,0 +1,72 @@ +"""Conflict-graph partitioning of a wave into parallel-safe batches.""" + +from misterdev.core.execution.wave_partition import partition_parallel_safe + + +def test_overlapping_files_land_in_different_batches(): + """Two tasks that declare a shared file are never in the same batch.""" + batches = partition_parallel_safe([("A", {"env.ts"}), ("B", {"env.ts"})]) + assert len(batches) == 2 + a = next(b for b in batches if "A" in b) + assert "B" not in a + + +def test_disjoint_tasks_share_a_batch(): + """Tasks with disjoint file sets run in parallel (one batch).""" + batches = partition_parallel_safe( + [("A", {"a.ts"}), ("B", {"b.ts"}), ("C", {"c.ts"})] + ) + assert batches == [["A", "B", "C"]] + + +def test_mixed_overlap_keeps_disjoint_parallel(): + """A conflicts with B on a shared file; C is disjoint from both, so C joins + A's batch and only B is pushed to a second sub-wave.""" + batches = partition_parallel_safe( + [("A", {"env.ts"}), ("B", {"env.ts"}), ("C", {"routes.ts"})] + ) + assert batches == [["A", "C"], ["B"]] + + +def test_empty_file_set_makes_no_claim(): + """A task with no declared files claims nothing and joins the first batch.""" + batches = partition_parallel_safe( + [("A", {"shared.ts"}), ("B", set()), ("C", {"shared.ts"})] + ) + # A and B share batch 0 (B claims nothing); C conflicts with A -> batch 1. + assert batches == [["A", "B"], ["C"]] + + +def test_transitive_chain_two_batches(): + """A-B share f1, B-C share f2, A and C are disjoint: A and C can run together, + only B is isolated.""" + batches = partition_parallel_safe( + [("A", {"f1"}), ("B", {"f1", "f2"}), ("C", {"f2"})] + ) + assert batches == [["A", "C"], ["B"]] + + +def test_three_way_shared_file_serializes_fully(): + """Three tasks all editing the same file must run in three separate batches.""" + batches = partition_parallel_safe( + [("A", {"g.ts"}), ("B", {"g.ts"}), ("C", {"g.ts"})] + ) + assert batches == [["A"], ["B"], ["C"]] + + +def test_empty_input(): + assert partition_parallel_safe([]) == [] + + +def test_order_preserved_and_all_items_present(): + items = [(f"T{i}", {f"file{i % 3}.ts"}) for i in range(9)] + batches = partition_parallel_safe(items) + flat = [t for b in batches for t in b] + assert sorted(flat) == sorted(t for t, _ in items) + # No batch contains two tasks sharing a file (file0/3/6, file1/4/7, ...). + fset = dict(items) + for batch in batches: + seen: set = set() + for t in batch: + assert seen.isdisjoint(fset[t]) + seen |= fset[t] diff --git a/tests/test_web_verify.py b/tests/test_web_verify.py index a6889e0..6b308c5 100644 --- a/tests/test_web_verify.py +++ b/tests/test_web_verify.py @@ -605,3 +605,32 @@ def test_live_web_runs_or_skips(tmp_path): pytest.skip(f"no browser available: {res.reason}") assert res.status == GREEN assert os.path.exists(res.evidence) + + +def test_start_server_does_not_block_past_deadline_and_no_leak(): + # A silent server that never signals readiness must not block readline past the + # deadline (which would leak the process past the gate's outer bound). + import threading + import time as _time + from pathlib import Path as _Path + + from misterdev.core.verification.web_verify import _start_server, _terminate + + result = {} + + def run(): + result["proc"] = _start_server( + _Path("."), "sleep 5", "NEVER_READY", _time.monotonic() + 0.3 + ) + + t = threading.Thread(target=run, daemon=True) + t.start() + t.join(timeout=3) + try: + assert not t.is_alive(), "_start_server blocked past its deadline (leak)" + proc = result.get("proc") + assert proc is not None + _terminate(proc) + assert proc.poll() is not None # terminated, not leaked + finally: + _terminate(result.get("proc")) diff --git a/tests/test_zero_test_gate_hard_rejected.py b/tests/test_zero_test_gate_hard_rejected.py new file mode 100644 index 0000000..7e690c4 --- /dev/null +++ b/tests/test_zero_test_gate_hard_rejected.py @@ -0,0 +1,60 @@ +"""T0.2 — a test gate that collects/executes zero tests is a hard reject. + +The acceptance layer (`GatesMixin._gate_accepts`) must not treat a test command +that exits 0 while running ZERO tests as a pass. A zero-test "green" greenlights +any edit while catching no regression — the canonical false-GREEN gate. The +zero-test signal machinery already exists in the validator (`gate_ran_no_tests` +paired with a parsed total of 0); this asserts `_gate_accepts` actually consults +it, and that genuine greens and the baseline-red path are unaffected. +""" + +import pytest + +from misterdev.task_executors.markdown_plan_executor.gates_mixin import GatesMixin + +# Runner-specific "zero tests executed" outputs, each paired with exit 0 (success). +ZERO_TEST_GREENS = [ + "collected 0 items\n\nno tests ran in 0.01s", # pytest + "no tests ran in 0.00s", # pytest (variant) + "Ran 0 tests in 0.000s\n\nOK", # stdlib unittest + "No tests found, exiting with code 0", # jest / vitest + "no test files\nok \tpkg\t0.001s", # go + "running 0 tests\n\ntest result: ok. 0 passed; 0 failed", # cargo +] + + +@pytest.mark.parametrize("output", ZERO_TEST_GREENS) +def test_zero_test_green_is_rejected(output): + accepted, _post = GatesMixin._gate_accepts(True, output, baseline_failures=0) + assert accepted is False, ( + "a test gate that ran zero tests must be rejected, not accepted as GREEN; " + f"output was {output!r}" + ) + + +def test_real_green_with_tests_still_accepted(): + accepted, post = GatesMixin._gate_accepts(True, "5 passed in 1.2s", 0) + assert accepted is True + assert post == 0 + + +def test_empty_output_green_is_not_falsely_rejected(): + # No zero-test signal present -> a plain success must remain accepted (we do + # not punish output formats we simply do not parse). + accepted, _ = GatesMixin._gate_accepts(True, "", 0) + assert accepted is True + + +def test_baseline_red_incremental_progress_unaffected(): + # Red run, parsed 2 failures, baseline 3 -> still accepted (no-worse rule). + accepted, post = GatesMixin._gate_accepts(False, "3 passed, 2 failed", 3) + assert accepted is True + assert post == 2 + + +def test_zero_test_green_rejected_even_under_red_baseline(): + # A zero-test green must not sneak through the baseline-red branch either. + accepted, _ = GatesMixin._gate_accepts( + True, "collected 0 items", baseline_failures=5 + ) + assert accepted is False