diff --git a/docs/release-qualification.md b/docs/release-qualification.md index 048655b5..3b17a3ca 100644 --- a/docs/release-qualification.md +++ b/docs/release-qualification.md @@ -241,9 +241,9 @@ This overlays only `campaign_adapter_argv` and `campaign_adapter_timeout_seconds Code Mower maintainers shipping a built-in adapter for a provider instead add `campaign_adapter_argv` (and optionally `campaign_adapter_timeout_seconds`) directly to the provider's `provider_config` in `src/code_mower/provider_registry.py`, using the same placeholders and contract described above. Most adopters do not need to touch this file. -Hosted / SaaS providers (`hosted_bridge`/`saas_event` driver: Devin, Cursor BugBot) dispatch via a GitHub issue comment instead of a local adapter: +Hosted / SaaS providers (`hosted_bridge`/`saas_event` driver: Devin, Cursor Cloud Agent) dispatch via a GitHub issue comment instead of a local adapter. Every hosted dispatch follows a closed five-check profile -- auth (dispatch token), installation (the provider App answers campaign comments, acknowledged via the lane's `campaign_transport_ready_env`), trigger (the provider's real builder trigger text), trusted responder (`bot_authors` allowlist), and result return (bounded response wait). Doctor reports each check independently, and a dry-run with an unverified App transport or result-return path reports the provider `unavailable` (`hosted_transport_unverified`) with the exact remediation instead of previewing it queued. Only an explicit `--apply` dispatches; silence past the deadline becomes `hosted_response_timeout` evidence, and only an explicit `--retry-provider` may dispatch again -- paid work is never retried automatically: -- Configure authentication tokens (`DEVIN_AUDIT_LABEL_TOKEN`, `CURSOR_BUGBOT_AUDIT_LABEL_TOKEN`, `GITHUB_TOKEN`) and supply `--issue ` plus `--repo-slug ` (at creation, or on the `resume`/`dispatch` that first needs it). Without both, the hosted provider stays `unavailable` and no comment is posted. The dry-run preview judges this prerequisite exactly as `--apply` does: a hosted provider with valid credentials but no issue number previews as `unavailable` with the bounded `missing_issue_number` error code and a next action naming `--issue`, rather than as queued and ready to dispatch. +- Configure the provider token (`DEVIN_AUDIT_LABEL_TOKEN` or `CURSOR_CLOUD_AGENT_AUDIT_LABEL_TOKEN`, with `GITHUB_TOKEN` as fallback) and supply `--issue ` plus `--repo-slug ` (at creation, or on the `resume`/`dispatch` that first needs it). Without both, the hosted provider stays `unavailable` and no comment is posted. The dry-run preview judges this prerequisite exactly as `--apply` does: a hosted provider with valid credentials but no issue number previews as `unavailable` with the bounded `missing_issue_number` error code and a next action naming `--issue`, rather than as queued and ready to dispatch. - The dispatch comment states exactly what will be accepted. For an upgrade campaign it carries the campaign's exact `starting_version` in both the machine-readable `code_mower.releaseCampaignDispatch.v1` marker and the human-facing instructions, so a remote runner never has to guess which starting version to qualify from. Cold-install (and `unknown`) campaigns have no starting version and omit the field. An upgrade campaign whose stored `starting_version` is missing is never dispatched at all: the provider stays `unavailable` with the bounded `campaign_identity_incomplete` error code and no comment is posted. - The provider's reply comment must embed a `CODE_MOWER_ADOPTION_RESULT` marker as a single-line HTML comment on a line of its own (``), wrapping schema `code_mower.releaseCampaignResult.v1` with `campaign_id`, `provider`, `release_tag`, and `idempotency_key` matching the original dispatch, plus a validated `adoption_result`. A bare or unbound result is ignored so a stale or unrelated comment can never be replayed as evidence. The embedded `adoption_result`'s own `qualification_context` and `starting_version` must also match the campaign's exactly, independent of the wrapper's idempotency key -- a cold-install result cannot complete an upgrade campaign, and an upgrade result from one starting version cannot complete a same-tag upgrade campaign from a different starting version. The marker line is matched end to end and its JSON is captured through the object's own final brace, so a literal `-->` inside a permitted string value cannot truncate an otherwise valid trusted result; a marker whose JSON is genuinely malformed is still ignored (fail-closed), never guessed at. - These identity fields are visible in the public dispatch comment, so binding alone does not prove authorship -- anyone could reply with a matching marker. A result marker is only ever accepted from a GitHub comment author present in the lane's `provider_config.bot_authors` list (and, if configured, the comma-separated login list in the environment variable named by `provider_config.bot_authors_env`). A lane with no trusted authors configured trusts nobody; an untrusted or spoofed author's comment is ignored and the provider keeps running. @@ -257,10 +257,11 @@ Cursor Cloud Agent is a hosted async builder using the `hosted_bridge` driver. - `CURSOR_CLOUD_AGENT_AUDIT_LABEL_TOKEN` (or `GITHUB_TOKEN` as fallback) for applying audit labels - `GITHUB_TOKEN` for posting dispatch comments - After verifying that the installed App answers campaign issue comments, set - `CODE_MOWER_CURSOR_CLOUD_AGENT_CAMPAIGN_TRANSPORT_READY=1`. Without it, doctor and - Board report the transport as unverified, but an explicit `--apply` may still - dispatch it under the response deadline below. Token presence alone proves - comment permission, not that the App supports this transport. + `CODE_MOWER_CURSOR_CLOUD_AGENT_CAMPAIGN_TRANSPORT_READY=1`. Without it, doctor + and the campaign dry-run report the transport as unverified (`unavailable` + with the exact remediation) before any dispatch; an explicit `--apply` may + still dispatch it under the response deadline below. Token presence alone + proves comment permission, not that the App supports this transport. **Trusted authors (default):** - `cursor[bot]` @@ -269,9 +270,12 @@ Cursor Cloud Agent is a hosted async builder using the `hosted_bridge` driver. **Environment override:** Set `CURSOR_CLOUD_AGENT_BOT_AUTHORS` to a comma-separated list of additional trusted GitHub logins. This extends (does not replace) the default trusted authors, allowing self-hosted or alternative Cursor integrations to be trusted. -**Trigger comments:** -- `@cursor run` -- `cursor run` +**Trigger comments (real builder contract):** +- `@cursor` + +The `@cursor` mention is the dispatch-lanes builder trigger (see +`docs/lanes/cursor.md`). Never use BugBot/reviewer trigger text (`bugbot run`, +`@cursor review`) for release qualification. **Role and capability:** - `role: builder` @@ -350,8 +354,9 @@ Devin is a hosted paid provider using the `hosted_bridge` driver. - `DEVIN_AUDIT_LABEL_TOKEN` (or `GITHUB_TOKEN` as fallback) for applying audit labels - `GITHUB_TOKEN` for posting dispatch comments - After verifying that the installed App answers campaign issue comments, set - `CODE_MOWER_DEVIN_CAMPAIGN_TRANSPORT_READY=1`. Without it, doctor and Board - report the transport as unverified, but an explicit `--apply` may still + `CODE_MOWER_DEVIN_CAMPAIGN_TRANSPORT_READY=1`. Without it, doctor and the + campaign dry-run report the transport as unverified (`unavailable` with the + exact remediation) before any dispatch; an explicit `--apply` may still dispatch it under the response deadline below. Token presence alone does not prove that the App supports this transport. diff --git a/src/code_mower/doctor_checks/adoption.py b/src/code_mower/doctor_checks/adoption.py index c8207c15..2382f0df 100644 --- a/src/code_mower/doctor_checks/adoption.py +++ b/src/code_mower/doctor_checks/adoption.py @@ -617,6 +617,8 @@ def check_adoption_campaign_readiness( _safe_error, _validate_adapter_argv_template, _validate_adapter_timeout, + hosted_dispatch_blockers, + hosted_dispatch_profile, resolve_supported_runtime, resolve_provider_lane, ) @@ -933,9 +935,19 @@ def check_adoption_campaign_readiness( transport_ready, transport_var = _check_hosted_transport( lane, env=current_env ) + # Closed dispatch profile: auth, installation, trigger, + # trusted responder, and result return are reported + # independently, so one verified dimension never masks another. + dispatch_profile = hosted_dispatch_profile(lane, env=current_env) + dispatch_blockers = hosted_dispatch_blockers(dispatch_profile) + dispatch_summary = { + name: bool(entry.get("ready")) for name, entry in dispatch_profile.items() + } has_repo = bool(repo_slug) structured_capability = check_structured_result_capability(canonical) - cmd_ready = bool(has_credentials and has_repo and transport_ready) + cmd_ready = bool( + has_credentials and has_repo and transport_ready and not dispatch_blockers + ) if cmd_ready: auth_state = "ready" elif not has_credentials: @@ -1006,6 +1018,8 @@ def check_adoption_campaign_readiness( "has_credentials": True, "transport_verified": False, "verification_variable": transport_var, + "dispatch_profile": dispatch_summary, + "dispatch_blockers": dispatch_blockers, "enabled": is_enabled, "actionable": is_enabled, "optional": not is_enabled, @@ -1025,6 +1039,55 @@ def check_adoption_campaign_readiness( ), ) ) + elif dispatch_blockers: + # Credentials and transport alone are not readiness: any + # failed closed dispatch-profile check warns with the bounded + # blocker names and their metadata-only remediations, so a + # lane missing its trigger, trusted responder allowlist, or + # result-return wait can never report PASS. + blocker_remediations = { + name: str(dispatch_profile.get(name, {}).get("remediation") or "") + for name in dispatch_blockers + } + detail = { + "provider": canonical, + "lane": lane.lane_id, + "driver": lane.driver, + "repo_slug": repo_slug, + "has_credentials": True, + "transport_verified": True, + "dispatch_profile": dispatch_summary, + "dispatch_blockers": dispatch_blockers, + "blocker_remediations": blocker_remediations, + "enabled": is_enabled, + "actionable": is_enabled, + "optional": not is_enabled, + } + if is_enabled: + detail["owner_action"] = True + checks.append( + DoctorCheck( + name="doctor.campaign.transport", + status=STATUS_WARN, + lane=canonical, + message=( + f"{canonical} hosted dispatch blocked: " + f"{', '.join(dispatch_blockers)}" + ), + detail=detail, + remediation=( + "; ".join( + blocker_remediations[name] + for name in dispatch_blockers + if blocker_remediations[name] + ) + or ( + f"Resolve {', '.join(dispatch_blockers)} for " + f"{canonical} before dispatching release qualification." + ) + ), + ) + ) else: checks.append( DoctorCheck( @@ -1039,6 +1102,7 @@ def check_adoption_campaign_readiness( "repo_slug": repo_slug, "has_credentials": True, "transport_verified": True, + "dispatch_profile": dispatch_summary, "enabled": is_enabled, }, ) diff --git a/src/code_mower/provider_registry.py b/src/code_mower/provider_registry.py index e235781d..836ef65d 100644 --- a/src/code_mower/provider_registry.py +++ b/src/code_mower/provider_registry.py @@ -314,7 +314,11 @@ def _maintained_campaign_adapter_argv(provider: str) -> tuple[str, ...]: provider_config={ "bot_authors": ("cursor[bot]", "cursor"), "bot_authors_env": "CURSOR_CLOUD_AGENT_BOT_AUTHORS", - "trigger_comments": ("@cursor run", "cursor run"), + # Real builder trigger contract: the `@cursor` mention dispatch + # comment (see docs/lanes/cursor.md and the dispatch-lanes + # workflow). Never BugBot/reviewer trigger text such as + # `bugbot run` or `@cursor review`. + "trigger_comments": ("@cursor",), "campaign_transport_ready_env": "CODE_MOWER_CURSOR_CLOUD_AGENT_CAMPAIGN_TRANSPORT_READY", "campaign_response_timeout_seconds": 3600, "rules_file": ".cursor/AGENT.md", diff --git a/src/code_mower/release_campaigns.py b/src/code_mower/release_campaigns.py index b3287517..650c2dc8 100644 --- a/src/code_mower/release_campaigns.py +++ b/src/code_mower/release_campaigns.py @@ -172,6 +172,7 @@ "github_dispatch_failed", "github_poll_unavailable", "hosted_response_timeout", + "hosted_transport_unverified", "python_runtime_unavailable", "unknown_provider", } @@ -429,18 +430,161 @@ def _check_hosted_transport( return ready, variable -def _hosted_response_timeout(lane: ProviderLane) -> int: +def _configured_hosted_response_timeout(lane: ProviderLane) -> int | None: value = lane.provider_config.get( "campaign_response_timeout_seconds", DEFAULT_HOSTED_RESPONSE_TIMEOUT_SECONDS, ) if isinstance(value, bool): - return DEFAULT_HOSTED_RESPONSE_TIMEOUT_SECONDS + return None try: parsed = int(value) except (TypeError, ValueError, OverflowError): - return DEFAULT_HOSTED_RESPONSE_TIMEOUT_SECONDS - return parsed if parsed > 0 else DEFAULT_HOSTED_RESPONSE_TIMEOUT_SECONDS + return None + return parsed if parsed > 0 else None + + +def _hosted_response_timeout(lane: ProviderLane) -> int: + return _configured_hosted_response_timeout(lane) or DEFAULT_HOSTED_RESPONSE_TIMEOUT_SECONDS + + +# Closed hosted dispatch profile: the five independent readiness checks a +# hosted (`hosted_bridge`/`saas_event`) release qualification must pass +# before a paid dispatch. Each check is judged on its own signal so one +# verified dimension can never mask another: +# * auth: a dispatch token is present (comment permission). +# * installation: the provider App is installed for the repo and has been +# seen answering campaign issue comments, acknowledged explicitly via the +# lane's `campaign_transport_ready_env`. Token presence alone proves +# nothing about the App. +# * trigger: the lane declares the provider's real builder trigger text, so +# the dispatch comment tells the remote runner exactly what to watch for. +# * trusted_responder: at least one GitHub login is trusted to post the +# adoption-result marker, so a reply can be authenticated. +# * result_return: the bounded response wait is configured, so provider +# silence degrades to `hosted_response_timeout` evidence instead of an +# unbounded hang. +HOSTED_DISPATCH_PROFILE_CHECKS = ( + "auth", + "installation", + "trigger", + "trusted_responder", + "result_return", +) + + +def hosted_dispatch_profile( + lane: ProviderLane, + *, + env: Mapping[str, str] | None = None, +) -> dict[str, dict[str, Any]]: + """Evaluate the closed hosted dispatch profile for one lane. + + Returns one ``{"ready": bool, "detail": str, "remediation": str}`` entry + per check in :data:`HOSTED_DISPATCH_PROFILE_CHECKS`. Details and + remediations are bounded metadata only: provider/lane names and + environment variable names, never secret values, paths, or output. + """ + current_env = os.environ if env is None else env + profile: dict[str, dict[str, Any]] = {} + + has_creds, missing_cred = _check_credentials(lane, env=current_env) + profile["auth"] = { + "ready": has_creds, + "detail": "dispatch token present" if has_creds else "dispatch token missing", + "remediation": ( + "" + if has_creds + else f"set {missing_cred} in the environment for {lane.provider} campaign dispatch" + ), + } + + transport_ready, transport_var = _check_hosted_transport(lane, env=current_env) + if not transport_var: + profile["installation"] = { + "ready": True, + "detail": "no separate installation acknowledgement configured", + "remediation": "", + } + else: + profile["installation"] = { + "ready": transport_ready, + "detail": ( + "provider App installation verified" + if transport_ready + else "provider App installation not verified" + ), + "remediation": ( + "" + if transport_ready + else ( + f"verify the {lane.provider} GitHub App answers campaign " + f"issue comments, then set {transport_var}=1" + ) + ), + } + + trigger_comments = tuple(lane.provider_config.get("trigger_comments") or ()) + trigger_ready = bool(trigger_comments) + profile["trigger"] = { + "ready": trigger_ready, + "detail": ( + "builder trigger configured" + if trigger_ready + else "no builder trigger configured" + ), + "remediation": ( + "" + if trigger_ready + else ( + f"configure the {lane.provider} builder trigger before dispatching " + f"release qualification" + ) + ), + } + + trusted_authors = _resolve_trusted_bot_authors(lane, env=current_env) + responder_ready = bool(trusted_authors) + profile["trusted_responder"] = { + "ready": responder_ready, + "detail": ( + "trusted responder allowlist configured" + if responder_ready + else "no trusted responder configured" + ), + "remediation": ( + "" + if responder_ready + else ( + f"configure bot_authors for {lane.provider} so adoption-result " + f"replies can be authenticated" + ) + ), + } + + timeout_ready = _configured_hosted_response_timeout(lane) is not None + profile["result_return"] = { + "ready": timeout_ready, + "detail": ( + "bounded result-return wait configured" + if timeout_ready + else "result-return wait is not a positive integer" + ), + "remediation": ( + "" + if timeout_ready + else ( + f"configure a positive campaign_response_timeout_seconds for " + f"{lane.provider} so silence becomes timeout evidence" + ) + ), + } + return profile + + +def hosted_dispatch_blockers(profile: Mapping[str, Mapping[str, Any]]) -> list[str]: + """Name the dispatch-profile checks that are not ready, in closed order.""" + return [name for name in HOSTED_DISPATCH_PROFILE_CHECKS if not profile.get(name, {}).get("ready")] def _response_deadline(started_at: str, timeout_seconds: int) -> str: @@ -1075,10 +1219,10 @@ def _post_trigger_comment( ) -> tuple[bool, dict[str, Any], str]: """Post the trigger command as a separate comment to actually start the provider. - For manually triggered hosted providers (Devin, Cursor BugBot), the dispatch - comment documents what to qualify, but the provider starts only when it sees - its configured trigger comment. Post that trigger as a plain comment body so - the provider will actually begin the qualification run. + For manually triggered hosted providers (Devin, Cursor Cloud Agent), the + dispatch comment documents what to qualify, but the provider starts only + when it sees its configured trigger comment. Post that trigger as a plain + comment body so the provider will actually begin the qualification run. """ if ( not repo_slug @@ -2530,8 +2674,25 @@ def dispatch_or_advance_campaign( # 4. Check capabilities and readiness cmd_found = _find_command(lane, which_fn=which_fn) has_creds, missing_cred = _check_credentials(lane, env=current_env) - transport_ready, transport_ready_var = _check_hosted_transport( - lane, env=current_env + transport_ready, _ = _check_hosted_transport(lane, env=current_env) + # Closed hosted dispatch profile: auth, installation, trigger, + # trusted responder, and result return are judged independently, so + # one verified dimension can never mask another. + dispatch_profile = ( + hosted_dispatch_profile(lane, env=current_env) + if lane.driver in {"hosted_bridge", "saas_event"} + else {} + ) + # Every failed check of the closed dispatch profile blocks the + # preview: auth, installation, trigger, trusted responder, and result + # return are judged independently, so one verified dimension can + # never mask another. Auth and issue prerequisites keep their more + # specific errors in the branches above; anything still failing here + # surfaces with its exact bounded remediation. + dispatch_blockers = ( + hosted_dispatch_blockers(dispatch_profile) + if lane.driver in {"hosted_bridge", "saas_event"} + else [] ) if lane.driver in {"hosted_bridge", "saas_event"}: provider_data["transport_verified"] = transport_ready @@ -2603,6 +2764,31 @@ def dispatch_or_advance_campaign( dry_run=True, error="missing issue number", ) + elif lane.driver in {"hosted_bridge", "saas_event"} and dispatch_blockers: + # A paid dispatch must never preview as queued while any + # closed dispatch-profile check fails: the preview judges + # exactly what --apply would need, so it reports unavailable + # with the exact bounded remediation before any dispatch. An + # explicit --apply may still dispatch (the operator's + # choice), under the bounded response deadline. + provider_data["state"] = "unavailable" + provider_data["error"] = _safe_error("hosted_transport_unverified") + blocker_remediations = [ + str(dispatch_profile.get(name, {}).get("remediation") or "") + for name in dispatch_blockers + ] + action = "; ".join(item for item in blocker_remediations if item) + if not action: + action = ( + f"resolve {', '.join(dispatch_blockers)} for {provider} " + f"before dispatching release qualification" + ) + detail = ( + f"{provider} hosted dispatch blocked: " + f"{', '.join(dispatch_blockers)}" + ) + if action: + detail = f"{detail}; {action}" else: provider_data["state"] = "queued" # A prerequisite recorded by an earlier preview (a missing @@ -2620,14 +2806,6 @@ def dispatch_or_advance_campaign( has_issue=has_issue, dry_run=True, ) - if ( - lane.driver in {"hosted_bridge", "saas_event"} - and not transport_ready - ): - detail = ( - f"{provider} transport is not independently verified; " - f"set {transport_ready_var}=1 after verification" - ) provider_data["next_action"] = action provider_data["next_detail"] = detail continue diff --git a/src/code_mower/templates/providers.yml b/src/code_mower/templates/providers.yml index 83739659..0a46c694 100644 --- a/src/code_mower/templates/providers.yml +++ b/src/code_mower/templates/providers.yml @@ -221,8 +221,7 @@ provider_templates: - "cursor" bot_authors_env: "CURSOR_CLOUD_AGENT_BOT_AUTHORS" trigger_comments: - - "@cursor run" - - "cursor run" + - "@cursor" campaign_transport_ready_env: "CODE_MOWER_CURSOR_CLOUD_AGENT_CAMPAIGN_TRANSPORT_READY" campaign_response_timeout_seconds: 3600 rules_file: ".cursor/AGENT.md" diff --git a/templates/providers.yml b/templates/providers.yml index 83739659..0a46c694 100644 --- a/templates/providers.yml +++ b/templates/providers.yml @@ -221,8 +221,7 @@ provider_templates: - "cursor" bot_authors_env: "CURSOR_CLOUD_AGENT_BOT_AUTHORS" trigger_comments: - - "@cursor run" - - "cursor run" + - "@cursor" campaign_transport_ready_env: "CODE_MOWER_CURSOR_CLOUD_AGENT_CAMPAIGN_TRANSPORT_READY" campaign_response_timeout_seconds: 3600 rules_file: ".cursor/AGENT.md" diff --git a/templates/providers/cursor_cloud_agent.yml b/templates/providers/cursor_cloud_agent.yml index acfc9eb5..d139701a 100644 --- a/templates/providers/cursor_cloud_agent.yml +++ b/templates/providers/cursor_cloud_agent.yml @@ -20,8 +20,7 @@ cursor_cloud_agent: - "cursor" bot_authors_env: "CURSOR_CLOUD_AGENT_BOT_AUTHORS" trigger_comments: - - "@cursor run" - - "cursor run" + - "@cursor" campaign_transport_ready_env: "CODE_MOWER_CURSOR_CLOUD_AGENT_CAMPAIGN_TRANSPORT_READY" campaign_response_timeout_seconds: 3600 rules_file: ".cursor/AGENT.md" diff --git a/tests/fixtures/hosted_dispatch/spoofed_responder.json b/tests/fixtures/hosted_dispatch/spoofed_responder.json new file mode 100644 index 00000000..9a787a5e --- /dev/null +++ b/tests/fixtures/hosted_dispatch/spoofed_responder.json @@ -0,0 +1,12 @@ +{ + "name": "spoofed_responder", + "description": "A comment carries an otherwise well-formed result marker but is posted by a login outside the lane's trusted responder allowlist. The marker must be ignored and the provider stays running.", + "provider": "cursor_cloud_agent", + "token_env": "CURSOR_CLOUD_AGENT_AUDIT_LABEL_TOKEN", + "transport_env": "CODE_MOWER_CURSOR_CLOUD_AGENT_CAMPAIGN_TRANSPORT_READY", + "trusted_author": "cursor[bot]", + "comment_author": "mallory", + "comment_has_bound_marker": true, + "expected_state": "running", + "expected_adoption_result": null +} diff --git a/tests/fixtures/hosted_dispatch/successful_return.json b/tests/fixtures/hosted_dispatch/successful_return.json new file mode 100644 index 00000000..ccaed983 --- /dev/null +++ b/tests/fixtures/hosted_dispatch/successful_return.json @@ -0,0 +1,14 @@ +{ + "name": "successful_return", + "description": "A trusted responder posts an identity-bound result marker whose embedded adoption result matches the campaign provider, release tag, package identity, qualification context, and starting version. The provider completes.", + "provider": "cursor_cloud_agent", + "token_env": "CURSOR_CLOUD_AGENT_AUDIT_LABEL_TOKEN", + "transport_env": "CODE_MOWER_CURSOR_CLOUD_AGENT_CAMPAIGN_TRANSPORT_READY", + "trusted_author": "cursor[bot]", + "comment_author": "cursor[bot]", + "comment_has_bound_marker": true, + "qualification_context": "cold_install", + "starting_version": "", + "expected_state": "complete", + "expected_outcome": "pass" +} diff --git a/tests/fixtures/hosted_dispatch/timeout.json b/tests/fixtures/hosted_dispatch/timeout.json new file mode 100644 index 00000000..28ca930c --- /dev/null +++ b/tests/fixtures/hosted_dispatch/timeout.json @@ -0,0 +1,16 @@ +{ + "name": "timeout", + "description": "Running paid dispatch passes its response deadline with no trusted result. The provider becomes unavailable with hosted_response_timeout evidence, and an ordinary resume must not redispatch; only an explicit retry may mutate.", + "provider": "cursor_cloud_agent", + "token_env": "CURSOR_CLOUD_AGENT_AUDIT_LABEL_TOKEN", + "transport_env": "CODE_MOWER_CURSOR_CLOUD_AGENT_CAMPAIGN_TRANSPORT_READY", + "token_present": true, + "transport_acknowledged": true, + "issue_present": true, + "repo_slug": "owner/repo", + "comments": [], + "response_deadline_at": "2026-01-01T00:00:00Z", + "expected_state": "unavailable", + "expected_error": "hosted_response_timeout", + "expected_resume_dispatch_calls": 0 +} diff --git a/tests/fixtures/hosted_dispatch/unavailable.json b/tests/fixtures/hosted_dispatch/unavailable.json new file mode 100644 index 00000000..e2a5c0b0 --- /dev/null +++ b/tests/fixtures/hosted_dispatch/unavailable.json @@ -0,0 +1,23 @@ +{ + "name": "unavailable", + "description": "App transport not verified: token and issue are present but the installation acknowledgement is absent. Doctor and campaign dry-run must report unavailable with the exact remediation before any dispatch.", + "provider": "devin", + "token_env": "DEVIN_AUDIT_LABEL_TOKEN", + "transport_env": "CODE_MOWER_DEVIN_CAMPAIGN_TRANSPORT_READY", + "token_present": true, + "transport_acknowledged": false, + "issue_present": true, + "repo_slug": "owner/repo", + "expected_profile": { + "auth": true, + "installation": false, + "trigger": true, + "trusted_responder": true, + "result_return": true + }, + "expected_state": "unavailable", + "expected_error": "hosted_transport_unverified", + "expected_transport_verified": false, + "expected_remediation_contains": "CODE_MOWER_DEVIN_CAMPAIGN_TRANSPORT_READY=1", + "expected_dispatch_calls": 0 +} diff --git a/tests/fixtures/hosted_dispatch/verified.json b/tests/fixtures/hosted_dispatch/verified.json new file mode 100644 index 00000000..2457a5e0 --- /dev/null +++ b/tests/fixtures/hosted_dispatch/verified.json @@ -0,0 +1,22 @@ +{ + "name": "verified", + "description": "Hosted dispatch profile fully verified: token present, App transport acknowledged, builder trigger declared, trusted responder allowlist configured, bounded result-return wait. Dry-run previews as queued with no dispatch side effect.", + "provider": "cursor_cloud_agent", + "token_env": "CURSOR_CLOUD_AGENT_AUDIT_LABEL_TOKEN", + "transport_env": "CODE_MOWER_CURSOR_CLOUD_AGENT_CAMPAIGN_TRANSPORT_READY", + "token_present": true, + "transport_acknowledged": true, + "issue_present": true, + "repo_slug": "owner/repo", + "expected_profile": { + "auth": true, + "installation": true, + "trigger": true, + "trusted_responder": true, + "result_return": true + }, + "expected_state": "queued", + "expected_error": "", + "expected_transport_verified": true, + "expected_dispatch_calls": 0 +} diff --git a/tests/test_hosted_dispatch_profile.py b/tests/test_hosted_dispatch_profile.py new file mode 100644 index 00000000..1f68202d --- /dev/null +++ b/tests/test_hosted_dispatch_profile.py @@ -0,0 +1,612 @@ +#!/usr/bin/env python3 +"""Bounded offline fixtures for hosted release-qualification dispatch. + +Covers the closed hosted dispatch profile (auth, installation, trigger, +trusted responder, result return) for the opt-in paid hosted providers +(Cursor Cloud Agent, Devin) without any network access: verified readiness, +unverified transport, response timeout, spoofed responder, and successful +result return. Every scenario is metadata-only -- no issue bodies, provider +output, auth output, paths, or secrets leave the test process. +""" + +from __future__ import annotations + +import dataclasses +import json +import sys +import tempfile +import unittest +from pathlib import Path +from typing import Any +from unittest import mock + +ROOT = Path(__file__).resolve().parents[1] +sys.path.insert(0, str(ROOT / "src")) + +from code_mower import release_campaigns +from code_mower.doctor_checks.adoption import check_adoption_campaign_readiness +from code_mower.provider_registry import REFERENCE_PROVIDERS + +FIXTURES_DIR = Path(__file__).resolve().parent / "fixtures" / "hosted_dispatch" + + +def load_fixture(name: str) -> dict[str, Any]: + with (FIXTURES_DIR / f"{name}.json").open("r", encoding="utf-8") as fh: + data = json.load(fh) + assert isinstance(data, dict) + return data + + +def _mock_adoption_result( + *, + release_tag: str = "v1.0.0", + provider: str = "cursor_cloud_agent", + qualification_context: str = "cold_install", + starting_version: str = "", + outcome: str = "pass", +) -> dict[str, Any]: + return { + "schema": "code_mower.adoptionResult.v1", + "timestamp_utc": "2026-09-04T08:00:00Z", + "release_tag": release_tag, + "package_identity": "code-mower", + "normalized_version": "1.0.0", + "qualification_context": qualification_context, + "starting_version": starting_version, + "ending_version": "1.0.0", + "provider": provider, + "executor": provider, + "host_class": "local", + "runtime_class": "python_3.12", + "execution_state": "executed", + "elapsed_seconds": 12.34, + "outcome": outcome, + "steps": [ + { + "id": "doctor", + "status": "pass", + "elapsed_seconds": 1.2, + "warning_count": 0, + "owner_action_count": 0, + }, + { + "id": "package_install", + "status": "pass", + "elapsed_seconds": 11.14, + "warning_count": 0, + "owner_action_count": 0, + }, + ], + } + + +def _fixture_env(fixture: dict[str, Any]) -> dict[str, str]: + env: dict[str, str] = {} + if fixture.get("token_present"): + env[str(fixture["token_env"])] = "token" + if fixture.get("transport_acknowledged"): + env[str(fixture["transport_env"])] = "1" + return env + + +class HostedDispatchProfileTests(unittest.TestCase): + """The five profile checks are judged independently from fixture inputs.""" + + def test_verified_fixture_reports_all_checks_ready(self) -> None: + fixture = load_fixture("verified") + lane = REFERENCE_PROVIDERS["cursor_cloud_agent"] + profile = release_campaigns.hosted_dispatch_profile( + lane, env=_fixture_env(fixture) + ) + self.assertEqual( + {name: entry["ready"] for name, entry in profile.items()}, + fixture["expected_profile"], + ) + self.assertEqual(release_campaigns.hosted_dispatch_blockers(profile), []) + + def test_unavailable_fixture_blocks_only_installation(self) -> None: + fixture = load_fixture("unavailable") + lane = REFERENCE_PROVIDERS["devin"] + profile = release_campaigns.hosted_dispatch_profile( + lane, env=_fixture_env(fixture) + ) + self.assertEqual( + {name: entry["ready"] for name, entry in profile.items()}, + fixture["expected_profile"], + ) + self.assertEqual(release_campaigns.hosted_dispatch_blockers(profile), ["installation"]) + remediation = profile["installation"]["remediation"] + self.assertIn(fixture["transport_env"], remediation) + # Bounded metadata only: no secret values, paths, or output. + self.assertNotIn("token", remediation) + + def test_result_return_rejects_non_positive_timeout(self) -> None: + from code_mower.provider_registry import LaneLabels, ProviderLane + + lane = ProviderLane( + lane_id="fake_hosted", + lane_type="audit", + driver="hosted_bridge", + provider="fake_hosted", + labels=LaneLabels(needs="needs-fake", done="fake-done", blocked="fake-blocked"), + trigger_policy="manual", + provider_config={ + "bot_authors": ("fake-bot[bot]",), + "trigger_comments": ("@fake run",), + "campaign_response_timeout_seconds": 0, + }, + ) + profile = release_campaigns.hosted_dispatch_profile( + lane, env={"GITHUB_TOKEN": "token"} + ) + self.assertTrue(profile["auth"]["ready"]) + self.assertTrue(profile["installation"]["ready"]) + self.assertTrue(profile["trigger"]["ready"]) + self.assertTrue(profile["trusted_responder"]["ready"]) + self.assertFalse(profile["result_return"]["ready"]) + self.assertIn("campaign_response_timeout_seconds", profile["result_return"]["remediation"]) + + def test_result_return_accepts_numeric_timeout_string(self) -> None: + base = REFERENCE_PROVIDERS["devin"] + config = dict(base.provider_config) + config["campaign_response_timeout_seconds"] = "3600" + lane = dataclasses.replace(base, provider_config=config) + profile = release_campaigns.hosted_dispatch_profile( + lane, + env={ + "DEVIN_AUDIT_LABEL_TOKEN": "token", + "CODE_MOWER_DEVIN_CAMPAIGN_TRANSPORT_READY": "1", + }, + ) + self.assertTrue(profile["result_return"]["ready"]) + self.assertEqual(release_campaigns._hosted_response_timeout(lane), 3600) + + def test_cursor_builder_trigger_is_the_real_mention_contract(self) -> None: + """Cursor Cloud Agent uses `@cursor`, never BugBot/reviewer trigger text.""" + lane = REFERENCE_PROVIDERS["cursor_cloud_agent"] + trigger_comments = tuple(lane.provider_config.get("trigger_comments") or ()) + self.assertEqual(trigger_comments, ("@cursor",)) + for trigger in trigger_comments: + lowered = trigger.lower() + self.assertNotIn("bugbot", lowered) + self.assertNotIn("review", lowered) + # The shipped templates agree with the registry. + for relative in ( + "templates/providers/cursor_cloud_agent.yml", + "templates/providers.yml", + "src/code_mower/templates/providers.yml", + ): + text = (ROOT / relative).read_text(encoding="utf-8") + self.assertIn('"@cursor"', text) + self.assertNotIn('"@cursor run"', text) + self.assertNotIn('"cursor run"', text) + + +class HostedDispatchFixtureCampaignTests(unittest.TestCase): + """Fixture-driven dry-run, timeout, spoof, and result-return behavior.""" + + def test_verified_fixture_previews_queued_without_dispatch(self) -> None: + fixture = load_fixture("verified") + with tempfile.TemporaryDirectory() as tmp: + campaigns_dir = Path(tmp) / "campaigns" + command_runner = mock.MagicMock() + gh_json_runner = mock.MagicMock() + ret = release_campaigns.campaign_command( + release_tag="v1.0.0", + package_spec="code-mower==1.0.0", + providers=[str(fixture["provider"])], + campaigns_dir=campaigns_dir, + repo_slug=str(fixture["repo_slug"]), + issue="42", + apply=False, + command_runner=command_runner, + gh_json_runner=gh_json_runner, + env=_fixture_env(fixture), + ) + self.assertEqual(ret, 0) + saved = release_campaigns.load_campaign_by_id("campaign-v1.0.0", campaigns_dir) + assert saved is not None + entry = saved["providers"][0] + self.assertEqual(entry["state"], fixture["expected_state"]) + self.assertEqual(entry["error"], fixture["expected_error"]) + self.assertEqual(entry["transport_verified"], fixture["expected_transport_verified"]) + self.assertEqual(command_runner.call_count, fixture["expected_dispatch_calls"]) + gh_json_runner.assert_not_called() + + def test_unavailable_fixture_reports_exact_remediation_before_dispatch(self) -> None: + fixture = load_fixture("unavailable") + with tempfile.TemporaryDirectory() as tmp: + campaigns_dir = Path(tmp) / "campaigns" + command_runner = mock.MagicMock() + gh_json_runner = mock.MagicMock() + ret = release_campaigns.campaign_command( + release_tag="v1.0.0", + package_spec="code-mower==1.0.0", + providers=[str(fixture["provider"])], + campaigns_dir=campaigns_dir, + repo_slug=str(fixture["repo_slug"]), + issue="42", + apply=False, + command_runner=command_runner, + gh_json_runner=gh_json_runner, + env=_fixture_env(fixture), + ) + self.assertEqual(ret, 0) + saved = release_campaigns.load_campaign_by_id("campaign-v1.0.0", campaigns_dir) + assert saved is not None + entry = saved["providers"][0] + self.assertEqual(entry["state"], fixture["expected_state"]) + self.assertEqual(entry["error"], fixture["expected_error"]) + self.assertIn(entry["error"], release_campaigns.SAFE_ERROR_CODES) + self.assertIn(str(fixture["expected_remediation_contains"]), entry["next_action"]) + self.assertIn(str(fixture["expected_remediation_contains"]), entry["next_detail"]) + self.assertNotIn("--apply", entry["next_action"]) + self.assertEqual(saved["status"], "unavailable") + self.assertEqual(command_runner.call_count, fixture["expected_dispatch_calls"]) + gh_json_runner.assert_not_called() + + def _running_campaign(self, campaigns_dir: Path, fixture: dict[str, Any]) -> dict[str, Any]: + campaign = release_campaigns.initialize_campaign( + release_tag="v1.0.0", + package_spec="code-mower==1.0.0", + providers=[str(fixture["provider"])], + repo_slug="owner/repo", + ) + provider = campaign.providers[0] + campaign.status = "running" + provider["state"] = "running" + provider["attempted_at"] = "2026-09-04T08:00:00Z" + provider["dispatched_at"] = "2026-09-04T08:00:00Z" + provider["trigger_posted"] = True + provider["response_deadline_at"] = "2030-01-01T00:00:00Z" + provider["dispatch_ref"] = {"issue_number": "99", "comment_posted": True} + release_campaigns.save_campaign(campaign, campaigns_dir) + return provider + + def _bound_marker_body( + self, + *, + campaign_id: str, + provider: str, + release_tag: str, + idempotency_key: str, + author_note: str = "Qualification complete.", + ) -> str: + adoption_res = _mock_adoption_result( + release_tag=release_tag, + provider=provider, + ) + wrapper = { + "schema": release_campaigns.RESULT_MARKER_SCHEMA, + "campaign_id": campaign_id, + "provider": provider, + "release_tag": release_tag, + "idempotency_key": idempotency_key, + "adoption_result": adoption_res, + } + return f"{author_note}\n\n" + + def test_timeout_fixture_records_evidence_and_never_redispatches(self) -> None: + fixture = load_fixture("timeout") + with tempfile.TemporaryDirectory() as tmp: + campaigns_dir = Path(tmp) / "campaigns" + provider = self._running_campaign(campaigns_dir, fixture) + provider["response_deadline_at"] = str(fixture["response_deadline_at"]) + campaign = release_campaigns.load_campaign_by_id("campaign-v1.0.0", campaigns_dir) + assert campaign is not None + campaign["providers"][0]["response_deadline_at"] = str( + fixture["response_deadline_at"] + ) + release_campaigns.save_campaign(campaign, campaigns_dir) + + command_runner = mock.MagicMock() + + def mock_gh_json(args, **kwargs): + return {"comments": list(fixture["comments"])}, "" + + ret = release_campaigns.campaign_command( + release_tag="v1.0.0", + campaigns_dir=campaigns_dir, + resume=True, + repo_slug="owner/repo", + command_runner=command_runner, + gh_json_runner=mock_gh_json, + env=_fixture_env(fixture), + ) + self.assertEqual(ret, 0) + saved = release_campaigns.load_campaign_by_id("campaign-v1.0.0", campaigns_dir) + assert saved is not None + entry = saved["providers"][0] + self.assertEqual(entry["state"], fixture["expected_state"]) + self.assertEqual(entry["error"], fixture["expected_error"]) + self.assertIn(entry["error"], release_campaigns.SAFE_ERROR_CODES) + self.assertIn("--retry-provider", entry["next_action"]) + # An ordinary resume records the timeout; it never redispatches + # paid work, and repeating it still dispatches nothing. + self.assertEqual( + command_runner.call_count, fixture["expected_resume_dispatch_calls"] + ) + ret = release_campaigns.campaign_command( + release_tag="v1.0.0", + campaigns_dir=campaigns_dir, + resume=True, + repo_slug="owner/repo", + command_runner=command_runner, + gh_json_runner=mock_gh_json, + env=_fixture_env(fixture), + ) + self.assertEqual(ret, 0) + self.assertEqual( + command_runner.call_count, fixture["expected_resume_dispatch_calls"] + ) + + def test_spoofed_responder_marker_is_ignored(self) -> None: + fixture = load_fixture("spoofed_responder") + with tempfile.TemporaryDirectory() as tmp: + campaigns_dir = Path(tmp) / "campaigns" + provider = self._running_campaign(campaigns_dir, fixture) + body = self._bound_marker_body( + campaign_id="campaign-v1.0.0", + provider=str(fixture["provider"]), + release_tag="v1.0.0", + idempotency_key=str(provider["idempotency_key"]), + ) + + def mock_gh_json(args, **kwargs): + return { + "comments": [ + {"author": {"login": str(fixture["comment_author"])}, "body": body} + ] + }, "" + + release_campaigns.campaign_command( + release_tag="v1.0.0", + campaigns_dir=campaigns_dir, + resume=True, + repo_slug="owner/repo", + command_runner=mock.MagicMock(), + gh_json_runner=mock_gh_json, + env=_fixture_env(fixture), + ) + saved = release_campaigns.load_campaign_by_id("campaign-v1.0.0", campaigns_dir) + assert saved is not None + entry = saved["providers"][0] + self.assertEqual(entry["state"], fixture["expected_state"]) + self.assertIsNone(entry["adoption_result"]) + + def test_successful_return_completes_from_trusted_responder(self) -> None: + fixture = load_fixture("successful_return") + with tempfile.TemporaryDirectory() as tmp: + campaigns_dir = Path(tmp) / "campaigns" + provider = self._running_campaign(campaigns_dir, fixture) + body = self._bound_marker_body( + campaign_id="campaign-v1.0.0", + provider=str(fixture["provider"]), + release_tag="v1.0.0", + idempotency_key=str(provider["idempotency_key"]), + ) + + def mock_gh_json(args, **kwargs): + return { + "comments": [ + {"author": {"login": str(fixture["comment_author"])}, "body": body} + ] + }, "" + + release_campaigns.campaign_command( + release_tag="v1.0.0", + campaigns_dir=campaigns_dir, + resume=True, + repo_slug="owner/repo", + command_runner=mock.MagicMock(), + gh_json_runner=mock_gh_json, + env=_fixture_env(fixture), + ) + saved = release_campaigns.load_campaign_by_id("campaign-v1.0.0", campaigns_dir) + assert saved is not None + entry = saved["providers"][0] + self.assertEqual(entry["state"], fixture["expected_state"]) + assert entry["adoption_result"] is not None + self.assertEqual( + entry["adoption_result"]["outcome"], fixture["expected_outcome"] + ) + + +class HostedDoctorProfileTests(unittest.TestCase): + """Doctor reports the closed profile and stays metadata-only.""" + + def test_doctor_transport_warn_carries_dispatch_profile(self) -> None: + with tempfile.TemporaryDirectory() as tmp: + checks = check_adoption_campaign_readiness( + config={}, + repo_root=Path(tmp), + repo_slug="owner/repo", + env={"DEVIN_AUDIT_LABEL_TOKEN": "token"}, + providers=["devin"], + ) + transport = [c for c in checks if c.name == "doctor.campaign.transport"] + self.assertEqual(len(transport), 1) + check = transport[0] + profile = check.detail.get("dispatch_profile") + self.assertIsInstance(profile, dict) + self.assertFalse(profile["installation"]) + self.assertTrue(profile["auth"]) + self.assertIn("installation", list(check.detail.get("dispatch_blockers") or [])) + self.assertNotIn("token", json.dumps(check.detail)) + + def test_doctor_verified_pass_carries_full_dispatch_profile(self) -> None: + with tempfile.TemporaryDirectory() as tmp: + checks = check_adoption_campaign_readiness( + config={}, + repo_root=Path(tmp), + repo_slug="owner/repo", + env={ + "DEVIN_AUDIT_LABEL_TOKEN": "token", + "CODE_MOWER_DEVIN_CAMPAIGN_TRANSPORT_READY": "1", + }, + providers=["devin"], + ) + creds = [c for c in checks if c.name == "doctor.campaign.credentials"] + self.assertEqual(len(creds), 1) + profile = creds[0].detail.get("dispatch_profile") + self.assertIsInstance(profile, dict) + self.assertTrue(all(profile.values())) + self.assertNotIn("token", json.dumps(creds[0].detail)) + + +def _devin_lane_without( + *, + trigger_comments: bool = True, + trusted_responders: bool = True, +): + """Return the devin lane minus trigger text and/or the responder allowlist.""" + base = REFERENCE_PROVIDERS["devin"] + config = dict(base.provider_config) + if not trigger_comments: + config["trigger_comments"] = () + if not trusted_responders: + config["bot_authors"] = () + config.pop("bot_authors_env", None) + return dataclasses.replace(base, provider_config=config) + + +def _verified_devin_env() -> dict[str, str]: + return { + "DEVIN_AUDIT_LABEL_TOKEN": "s3cret-token-value", + "CODE_MOWER_DEVIN_CAMPAIGN_TRANSPORT_READY": "1", + } + + +class HostedDispatchBlockerTests(unittest.TestCase): + """Trigger and trusted-responder blockers fail closed in dry-run and doctor.""" + + def _dry_run_devin(self, lane, env) -> dict[str, Any]: + with tempfile.TemporaryDirectory() as tmp: + campaigns_dir = Path(tmp) / "campaigns" + command_runner = mock.MagicMock() + gh_json_runner = mock.MagicMock() + with mock.patch.dict(release_campaigns.REFERENCE_PROVIDERS, {"devin": lane}): + ret = release_campaigns.campaign_command( + release_tag="v1.0.0", + package_spec="code-mower==1.0.0", + providers=["devin"], + campaigns_dir=campaigns_dir, + repo_slug="owner/repo", + issue="42", + apply=False, + command_runner=command_runner, + gh_json_runner=gh_json_runner, + env=env, + ) + self.assertEqual(ret, 0) + saved = release_campaigns.load_campaign_by_id( + "campaign-v1.0.0", campaigns_dir + ) + assert saved is not None + command_runner.assert_not_called() + gh_json_runner.assert_not_called() + return saved + + def _assert_blocked_dry_run( + self, saved: dict[str, Any], *, blocker: str, remediation_marker: str + ) -> None: + entry = saved["providers"][0] + self.assertEqual(entry["state"], "unavailable") + self.assertEqual(entry["error"], "hosted_transport_unverified") + self.assertIn(entry["error"], release_campaigns.SAFE_ERROR_CODES) + self.assertIn(blocker, entry["next_detail"]) + self.assertIn(remediation_marker, entry["next_action"]) + self.assertIn(remediation_marker, entry["next_detail"]) + self.assertNotIn("--apply", entry["next_action"]) + self.assertNotIn("--apply", entry["next_detail"]) + self.assertEqual(saved["status"], "unavailable") + # Bounded metadata only: no secret values, paths, or output. + self.assertNotIn("s3cret-token-value", json.dumps(saved)) + + def test_dry_run_trigger_blocker_is_unavailable(self) -> None: + lane = _devin_lane_without(trigger_comments=False) + profile = release_campaigns.hosted_dispatch_profile( + lane, env=_verified_devin_env() + ) + self.assertEqual(release_campaigns.hosted_dispatch_blockers(profile), ["trigger"]) + saved = self._dry_run_devin(lane, _verified_devin_env()) + self._assert_blocked_dry_run( + saved, blocker="trigger", remediation_marker="builder trigger" + ) + + def test_dry_run_trusted_responder_blocker_is_unavailable(self) -> None: + lane = _devin_lane_without(trusted_responders=False) + profile = release_campaigns.hosted_dispatch_profile( + lane, env=_verified_devin_env() + ) + self.assertEqual( + release_campaigns.hosted_dispatch_blockers(profile), ["trusted_responder"] + ) + saved = self._dry_run_devin(lane, _verified_devin_env()) + self._assert_blocked_dry_run( + saved, blocker="trusted_responder", remediation_marker="bot_authors" + ) + + def test_dry_run_reports_every_blocker_remediation(self) -> None: + lane = _devin_lane_without( + trigger_comments=False, + trusted_responders=False, + ) + saved = self._dry_run_devin(lane, _verified_devin_env()) + entry = saved["providers"][0] + for blocker in ("trigger", "trusted_responder"): + self.assertIn(blocker, entry["next_detail"]) + for remediation_marker in ("builder trigger", "bot_authors"): + self.assertIn(remediation_marker, entry["next_action"]) + self.assertIn(remediation_marker, entry["next_detail"]) + + def _doctor_checks(self, lane) -> tuple: + with tempfile.TemporaryDirectory() as tmp: + with mock.patch.dict(release_campaigns.REFERENCE_PROVIDERS, {"devin": lane}): + return check_adoption_campaign_readiness( + config={}, + repo_root=Path(tmp), + repo_slug="owner/repo", + env=_verified_devin_env(), + providers=["devin"], + ) + + def _assert_blocked_doctor( + self, checks: tuple, *, blocker: str, remediation_marker: str + ) -> None: + transport = [c for c in checks if c.name == "doctor.campaign.transport"] + self.assertEqual(len(transport), 1) + check = transport[0] + self.assertEqual(check.status, "warn") + self.assertIn(blocker, list(check.detail.get("dispatch_blockers") or [])) + self.assertIn( + remediation_marker, + json.dumps(check.detail.get("blocker_remediations") or {}), + ) + self.assertIn(remediation_marker, check.remediation) + # No PASS for the blocked lane, and the aggregate never calls it ready. + passes = [ + c + for c in checks + if c.name == "doctor.campaign.credentials" and c.lane == "devin" + ] + self.assertEqual(passes, []) + readiness = next(c for c in checks if c.name == "doctor.campaign.readiness") + self.assertNotIn("devin", readiness.detail.get("ready_providers", [])) + self.assertNotIn("s3cret-token-value", json.dumps(checks, default=str)) + + def test_doctor_trigger_blocker_warns_without_pass(self) -> None: + checks = self._doctor_checks(_devin_lane_without(trigger_comments=False)) + self._assert_blocked_doctor( + checks, blocker="trigger", remediation_marker="builder trigger" + ) + + def test_doctor_trusted_responder_blocker_warns_without_pass(self) -> None: + checks = self._doctor_checks(_devin_lane_without(trusted_responders=False)) + self._assert_blocked_doctor( + checks, blocker="trusted_responder", remediation_marker="bot_authors" + ) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/test_release_campaigns.py b/tests/test_release_campaigns.py index 359cbe35..e54de646 100644 --- a/tests/test_release_campaigns.py +++ b/tests/test_release_campaigns.py @@ -2204,7 +2204,7 @@ def test_poll_requires_bound_comment_marker(self) -> None: """A bare adoptionResult JSON with no identity-bound wrapper is not accepted.""" with tempfile.TemporaryDirectory() as tmp: campaigns_dir = Path(tmp) / "campaigns" - adoption_res = _mock_adoption_result(release_tag="v1.0.0", provider="cursor_bugbot", outcome="pass") + adoption_res = _mock_adoption_result(release_tag="v1.0.0", provider="cursor_cloud_agent", outcome="pass") unbound_marker = f"" def mock_gh_json(args, **kwargs): @@ -2333,7 +2333,7 @@ def test_poll_rejects_upgrade_result_from_wrong_starting_version_despite_matchin adoption_res = _mock_adoption_result_full( release_tag="v2.0.0", normalized_version="2.0.0", - provider="cursor_bugbot", + provider="cursor_cloud_agent", qualification_context="upgrade", starting_version="0.9.0", ending_version="2.0.0", @@ -2342,7 +2342,7 @@ def test_poll_rejects_upgrade_result_from_wrong_starting_version_despite_matchin wrapper = { "schema": release_campaigns.RESULT_MARKER_SCHEMA, "campaign_id": campaign.campaign_id, - "provider": "cursor_bugbot", + "provider": "cursor_cloud_agent", "release_tag": "v2.0.0", "idempotency_key": idempotency_key, "adoption_result": adoption_res, @@ -2387,7 +2387,7 @@ def test_poll_rejects_cold_install_result_for_upgrade_campaign(self) -> None: adoption_res = _mock_adoption_result_full( release_tag="v2.0.0", normalized_version="2.0.0", - provider="cursor_bugbot", + provider="cursor_cloud_agent", qualification_context="cold_install", starting_version="", ending_version="2.0.0", @@ -2396,7 +2396,7 @@ def test_poll_rejects_cold_install_result_for_upgrade_campaign(self) -> None: wrapper = { "schema": release_campaigns.RESULT_MARKER_SCHEMA, "campaign_id": campaign.campaign_id, - "provider": "cursor_bugbot", + "provider": "cursor_cloud_agent", "release_tag": "v2.0.0", "idempotency_key": idempotency_key, "adoption_result": adoption_res, @@ -2945,11 +2945,11 @@ def test_cursor_cloud_agent_dispatch_posts_trigger_comment(self) -> None: trigger_body = bodies[1] # Dispatch body should document the trigger commands - self.assertIn("@cursor run", dispatch_body) + self.assertIn("@cursor", dispatch_body) # The actionable command stays first; the hidden marker makes a # crash-after-post retry externally idempotent. - self.assertEqual(trigger_body.splitlines()[0], "@cursor run") + self.assertEqual(trigger_body.splitlines()[0], "@cursor") self.assertIn("CODE_MOWER_RELEASE_TRIGGER", trigger_body) def test_failed_trigger_post_retries_on_resume(self) -> None: @@ -3072,7 +3072,7 @@ def mock_gh_json(args, **kwargs): # Should have posted exactly 1 trigger (no redispatch) self.assertEqual(len(bodies), 1) - self.assertIn("@cursor run", bodies[0]) + self.assertIn("@cursor", bodies[0]) self.assertNotIn("CODE_MOWER_RELEASE_CAMPAIGN", bodies[0]) resumed = release_campaigns.load_campaign_by_id("campaign-v1.0.0", campaigns_dir) @@ -3210,7 +3210,7 @@ def test_resume_reconciles_posted_trigger_marker_without_reposting(self) -> None "comments": [ { "author": {"login": "cursor[bot]"}, - "body": "@cursor run\n\n" + "body": "@cursor\n\n" f"", } ] @@ -3264,7 +3264,7 @@ def test_dispatch_nonce_cannot_forge_trigger_reconciliation(self) -> None: { "comments": [ { - "body": "@cursor run\n\n" + "body": "@cursor\n\n" f"" } ] @@ -3275,7 +3275,7 @@ def test_dispatch_nonce_cannot_forge_trigger_reconciliation(self) -> None: ) self.assertEqual(len(bodies), 1) - self.assertIn("@cursor run", bodies[0]) + self.assertIn("@cursor", bodies[0]) self.assertIn("private-trigger-key", bodies[0]) self.assertNotIn("public-dispatch-key", bodies[0]) resumed = release_campaigns.load_campaign_by_id( @@ -4135,31 +4135,54 @@ def test_hosted_dry_run_with_issue_is_queued(self) -> None: ) self._assert_no_dispatch(entry, command_runner, gh_json_runner, adapter_runner) - def test_hosted_transport_is_reported_unverified_without_blocking_preview(self) -> None: - with tempfile.TemporaryDirectory() as tmp: - campaigns_dir = Path(tmp) / "campaigns" - ret = release_campaigns.campaign_command( - release_tag="v1.0.0", - package_spec="code-mower==1.0.0", - providers=("devin",), - campaigns_dir=campaigns_dir, - repo_slug="owner/repo", - issue="42", - env={"DEVIN_AUDIT_LABEL_TOKEN": "token"}, - ) + def test_hosted_transport_unverified_is_unavailable_before_dispatch(self) -> None: + """An unverified App transport never previews as queued. - self.assertEqual(ret, 0) - saved = release_campaigns.load_campaign_by_id( - "campaign-v1.0.0", campaigns_dir - ) - assert saved is not None - entry = saved["providers"][0] - self.assertEqual(entry["state"], "queued") - self.assertFalse(entry["transport_verified"]) - self.assertIn( - "CODE_MOWER_DEVIN_CAMPAIGN_TRANSPORT_READY=1", - entry["next_detail"], - ) + Credentials plus an issue number are not dispatch readiness for paid + hosted work: without the explicit transport acknowledgement the + dry-run reports unavailable with the exact remediation, before any + dispatch. An explicit --apply remains the operator's own choice. + """ + for provider, token_env, transport_var in ( + ("devin", "DEVIN_AUDIT_LABEL_TOKEN", "CODE_MOWER_DEVIN_CAMPAIGN_TRANSPORT_READY"), + ( + "cursor_cloud_agent", + "CURSOR_CLOUD_AGENT_AUDIT_LABEL_TOKEN", + "CODE_MOWER_CURSOR_CLOUD_AGENT_CAMPAIGN_TRANSPORT_READY", + ), + ): + with self.subTest(provider=provider), tempfile.TemporaryDirectory() as tmp: + campaigns_dir = Path(tmp) / "campaigns" + command_runner = mock.MagicMock() + gh_json_runner = mock.MagicMock() + ret = release_campaigns.campaign_command( + release_tag="v1.0.0", + package_spec="code-mower==1.0.0", + providers=(provider,), + campaigns_dir=campaigns_dir, + repo_slug="owner/repo", + issue="42", + env={token_env: "token"}, + command_runner=command_runner, + gh_json_runner=gh_json_runner, + ) + + self.assertEqual(ret, 0) + saved = release_campaigns.load_campaign_by_id( + "campaign-v1.0.0", campaigns_dir + ) + assert saved is not None + entry = saved["providers"][0] + self.assertEqual(entry["state"], "unavailable") + self.assertEqual(entry["error"], "hosted_transport_unverified") + self.assertIn(entry["error"], release_campaigns.SAFE_ERROR_CODES) + self.assertFalse(entry["transport_verified"]) + self.assertIn(f"{transport_var}=1", entry["next_action"]) + self.assertIn(f"{transport_var}=1", entry["next_detail"]) + self.assertNotIn("--apply", entry["next_action"]) + self.assertEqual(saved["status"], "unavailable") + command_runner.assert_not_called() + gh_json_runner.assert_not_called() def test_hosted_dry_run_without_repo_slug_is_unavailable(self) -> None: """An issue number with no repo slug addresses nothing, exactly as under --apply.""" @@ -4448,7 +4471,10 @@ def test_a_mixed_preview_still_advertises_apply(self) -> None: repo_slug="owner/repo", issue="42", which_fn=lambda cmd: "/bin/aider" if cmd == "aider" else None, - env={"DEVIN_AUDIT_LABEL_TOKEN": "token"}, + env={ + "DEVIN_AUDIT_LABEL_TOKEN": "token", + "CODE_MOWER_DEVIN_CAMPAIGN_TRANSPORT_READY": "1", + }, ) states = {p["provider"]: p["state"] for p in saved["providers"]} self.assertEqual(states, {"devin": "queued", "aider": "unavailable"}) @@ -4834,6 +4860,45 @@ def test_cli_rejects_duplicate_providers_without_creating_a_campaign(self) -> No cmd_runner_mock.assert_not_called() self.assertEqual(release_campaigns.list_campaigns(campaigns_dir), []) + def test_cli_rejects_reviewer_only_providers_without_creating_a_campaign(self) -> None: + """CLI-facing: a reviewer-only alias is a bounded error, not a campaign. + + Every reviewer-only spelling (`cursor_bugbot`, `grok_bot`, + `cursor_grok_bot`) must fail closed at the command boundary with no + traceback, no runner side effect, and no campaign state file -- while + the builder identity (`cursor_cloud_agent`) still creates a campaign. + """ + for alias in ("cursor_bugbot", "grok_bot", "cursor_grok_bot"): + with self.subTest(provider=alias), tempfile.TemporaryDirectory() as tmp: + campaigns_dir = Path(tmp) / "campaigns" + adapter_mock = mock.MagicMock() + cmd_runner_mock = mock.MagicMock() + gh_json_mock = mock.MagicMock() + + stderr = io.StringIO() + with contextlib.redirect_stderr(stderr): + ret = release_campaigns.campaign_command( + release_tag="v1.0.0", + package_spec="code-mower==1.0.0", + providers=[alias], + campaigns_dir=campaigns_dir, + apply=True, + adapter_runner=adapter_mock, + command_runner=cmd_runner_mock, + gh_json_runner=gh_json_mock, + ) + + self.assertEqual(ret, 1) + self.assertIn("review-only lane", stderr.getvalue()) + self.assertNotIn("Traceback", stderr.getvalue()) + adapter_mock.assert_not_called() + cmd_runner_mock.assert_not_called() + gh_json_mock.assert_not_called() + self.assertEqual(release_campaigns.list_campaigns(campaigns_dir), []) + self.assertFalse( + (campaigns_dir / "campaign-v1.0.0.json").exists() + ) + class CampaignRepoSlugSupplyTests(unittest.TestCase): """A campaign created without a repo slug can be completed with one later.