Skip to content

feat!: model switching for /spec + legacy-systems cleanup (v9.0.0)#149

Merged
maxritter merged 3 commits into
mainfrom
dev
May 22, 2026
Merged

feat!: model switching for /spec + legacy-systems cleanup (v9.0.0)#149
maxritter merged 3 commits into
mainfrom
dev

Conversation

@maxritter

@maxritter maxritter commented May 22, 2026

Copy link
Copy Markdown
Owner

Release: v9.0.0 — Model switching for /spec + legacy-systems cleanup

This is a major-version release. Headline change: /spec now pauses after plan approval so you can pick a different model for implementation. Supporting that headline is a broad cleanup — the old per-skill model routing table, the credential scanner, and the legacy context-mode plugin are all gone.

Two commits, intentionally split for changelog clarity:

  • feat: — the new pause-and-resume behaviour (additive)
  • feat!: — the breaking-change cleanup that the new behaviour enables

1. feat: pause /spec after planning for a model switch

After plan approval, /spec ends the planning turn and waits so you can switch models for implementation. Two ways to resume:

  • Path A — switch in place. /model sonnet[1m] then any prompt (e.g. continue). Same session, planning context carries over. Claude Code asks to confirm the model switch — the existing context is re-sent to Sonnet on the next turn.
  • Path B — clean start. /clear, then /spec docs/plans/<plan>.md. Fresh session, no context carry. The dispatcher sees the approved plan and routes straight into implementation.

A new UserPromptSubmit hook (spec_handoff_resume) detects the handoff sentinel and dispatches the next prompt directly to spec-implement, so no /spec re-invocation is needed on Path A.

spec-mode-guard now hard-blocks /spec on non-Opus models for the planning leg — planning is where reasoning quality dominates outcome. Resuming an existing plan (/spec docs/plans/<plan>.md) is allowed on any model.

The Model Switching toggle in Console Settings → Automation controls whether the workflow pauses or runs continuously. Turn it off to stay on one model end-to-end (also the right setting for headless / CI).

Stop guard now leaves the handoff sentinel intact across the Stop event so the prompt hook can consume it; session_clear cleans up the sentinel when the user runs /clear.

2. feat!: collapse model selection to /model and prune legacy systems

Pilot no longer manages a per-skill / per-agent model routing table. Claude Code's /model is the single switch for every workflow. The Settings page's Models and Security sections, the Cost Optimization copy that named specific models, and the per-skill model: frontmatter on Pilot-shipped skills are all gone.

Config migration v12 strips the dead keys from ~/.pilot/config.json on first launch (model, skills, agents, extendedContext, extendedContextOverrides) and writes a pre-migration snapshot to ~/.pilot/config.json.bak.v11 once per machine. The new specWorkflow.modelSwitch toggle is seeded to true.

Credential / secret scanner hooks deleted along with the Console's Security tab and the security-credentials rule. The legacy mksglu/context-mode MCP plugin (auto-installed by older Pilot versions) is auto-uninstalled on upgrade, together with its orphan SessionStart hook in ~/.claude/settings.json.

Installer dependency output now distinguishes installed / updated / removed / unchanged. Re-runs no longer show a noisy ✓ X installed for every tool that was already there — only steps that actually did something report.

A one-time launcher notice infrastructure (launcher/notice.py) announces the rework on first launch after upgrade. The announcement is sentinel-gated and won't repeat.

Doc reorg: Permission Modes moved from Getting Started → Features. The standalone Security page is gone. Model Routing was rewritten to a single-page walkthrough with the two resume paths laid out explicitly. The marketing site's WhatsInside pillars were rebalanced — "Hooks, Quality & Security" became "Hooks & Quality" (security tile dropped), Cost Optimization no longer hardcodes model names, and card lengths were tightened so the grid reads consistently.


Breaking changes for users

  • Pilot no longer manages model selection. Users with model, skills, agents, extendedContext, or extendedContextOverrides keys in ~/.pilot/config.json will see those keys stripped on first launch. The pre-migration JSON is preserved at ~/.pilot/config.json.bak.v11. Pick your model with Claude Code's /model instead. /spec requires Opus for the planning leg (spec-mode-guard hard-blocks Sonnet).
  • The credential scanner and secret scanner hooks have been removed, along with the security-credentials rule, the Console's Security settings tab, and docs/features/security.md. Use Claude Code's built-in safeguards and project-level conventions for secret handling.
  • The legacy mksglu/context-mode Claude plugin (auto-installed by older Pilot versions) is auto-uninstalled on first launch after upgrade. Its orphan SessionStart hook in ~/.claude/settings.json is also removed.

Migration notes

No user action required for the model / config migration — it runs automatically on first launch and the safety snapshot at ~/.pilot/config.json.bak.v11 means hand-picked model IDs are recoverable.

If you relied on the credential scanner: it's gone with no in-product replacement. Local pre-commit hooks (e.g. git-secrets, trufflehog, gitleaks) are the recommended path.


Verification

  • All Python unit tests pass: 2149 passed, 9 skipped (installer, launcher, pilot/hooks, benchmark).
  • All Console tests pass: 1522 passed, 3 skipped.
  • TypeScript typecheck clean (bun run typecheck).
  • Console production build clean (npm run build regenerates bundled artifacts, staged in commit 2).
  • ruff check and basedpyright clean across modified Python files.

Summary by CodeRabbit

  • New Features

    • Added "Reset & Refresh" maintenance workflow to restore performance after extended use.
    • Added Model Switching pause/resume behavior (can be toggled in Console settings).
  • Documentation

    • Updated Console, Model Switching, Context Optimization, Hooks, MCP servers, installer, and FAQ guidance.
    • Removed Security Scanner documentation.
  • Chores

    • Installer and dependency list updated to Semble/RTK/CodeGraph and related tooling.
    • Installer now creates a pre-migration backup and seeds the Model Switching setting during config migration.

Review Change Stack

maxritter added 2 commits May 22, 2026 10:37
After plan approval, `/spec` now ends the planning turn and waits so you
can switch models for implementation. Two ways to resume:

  Path A — switch in place
    `/model sonnet[1m]` then any prompt (e.g. `continue`). Same session,
    planning context carries over. Claude Code asks to confirm the model
    switch — the existing context is re-sent to Sonnet on the next turn.

  Path B — clean start
    `/clear`, then `/spec docs/plans/<plan>.md`. Fresh session, no context
    carry. The dispatcher sees the approved plan and routes straight into
    implementation.

A new UserPromptSubmit hook (`spec_handoff_resume`) detects the handoff
sentinel and dispatches the next prompt directly to `spec-implement`, so
no `/spec` re-invocation is needed on Path A.

`spec-mode-guard` now hard-blocks `/spec` on non-Opus models for the
planning leg — planning is where reasoning quality dominates outcome.
Resuming an existing plan (`/spec docs/plans/<plan>.md`) is allowed on
any model.

The Model Switching toggle in Console Settings → Automation controls
whether the workflow pauses or runs continuously. Turn it off to stay on
one model end-to-end (also the right setting for headless / CI).

Stop guard now leaves the handoff sentinel intact across the Stop event
so the prompt hook can consume it; `session_clear` cleans up the sentinel
when the user runs `/clear`.
Pilot no longer manages a per-skill / per-agent model routing table.
Claude Code's `/model` is the single switch for every workflow. The
Settings page's Models and Security sections, the Cost Optimization
copy that named specific models, and the per-skill `model:` frontmatter
on Pilot-shipped skills are all gone.

Config migration v12 strips the dead keys from ~/.pilot/config.json on
first launch (`model`, `skills`, `agents`, `extendedContext`,
`extendedContextOverrides`) and writes a pre-migration snapshot to
~/.pilot/config.json.bak.v11 once per machine. The new
`specWorkflow.modelSwitch` toggle is seeded to true.

Credential / secret scanner hooks deleted along with the Console's
Security tab and the security-credentials rule. The legacy
mksglu/context-mode MCP plugin (auto-installed by older Pilot versions)
is auto-uninstalled on upgrade, together with its orphan SessionStart
hook in ~/.claude/settings.json.

Installer dependency output now distinguishes installed / updated /
removed / unchanged. Re-runs no longer show a noisy "✓ X installed" for
every tool that was already there — only steps that actually did
something report.

A one-time launcher notice infrastructure (`launcher/notice.py`)
announces the rework on first launch after upgrade. The announcement
is sentinel-gated and won't repeat.

Permission Modes moved from Getting Started to Features in the docs
sidebar; the standalone Security page is gone. Model Routing was
rewritten to a single-page walkthrough with the two resume paths laid
out explicitly. The marketing site's WhatsInside pillars were rebalanced
— "Hooks, Quality & Security" became "Hooks & Quality" (security tile
dropped), Cost Optimization no longer hardcodes model names, and card
lengths were tightened so the grid reads consistently.

BREAKING CHANGE: Pilot no longer manages model selection. Users with
`model`, `skills`, `agents`, `extendedContext`, or
`extendedContextOverrides` keys in ~/.pilot/config.json will see those
keys stripped on first launch. The pre-migration JSON is preserved at
~/.pilot/config.json.bak.v11. Pick your model with Claude Code's
`/model` instead. `/spec` requires Opus for the planning leg
(spec-mode-guard hard-blocks Sonnet).

BREAKING CHANGE: The credential scanner and secret scanner hooks have
been removed, along with the `security-credentials` rule, the Console's
Security settings tab, and the `docs/features/security.md` page. Use
Claude Code's built-in safeguards and project-level conventions for
secret handling.

BREAKING CHANGE: The legacy mksglu/context-mode Claude plugin
(auto-installed by older Pilot versions) is auto-uninstalled on first
launch after upgrade. Its orphan SessionStart hook in
~/.claude/settings.json is also removed.
@vercel

vercel Bot commented May 22, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
pilot-shell Ignored Ignored Preview May 22, 2026 9:03am

Request Review

@coderabbitai

coderabbitai Bot commented May 22, 2026

Copy link
Copy Markdown

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: e9835a2d-7ac1-4a15-91a6-ddbda16b659d

📥 Commits

Reviewing files that changed from the base of the PR and between 63e46e9 and 5afbf31.

📒 Files selected for processing (6)
  • README.md
  • docs/docusaurus/docs/features/console.md
  • docs/docusaurus/docs/getting-started/installation.md
  • docs/site/src/components/ConsoleSection.tsx
  • installer/steps/dependencies.py
  • installer/tests/unit/steps/test_dependencies.py

Walkthrough

This PR shifts Pilot Shell from a security-scanner/context-mode focus to a model-switching-first workflow: adds config v12 migration (prunes legacy model keys, seeds modelSwitch), removes deprecated claude artifacts, refactors installer/plugin install outcomes and cleanup, updates install/uninstall scripts, and refreshes docs/site content including Reset & Refresh guidance.

Changes

Config Migration & Data Schema

Layer / File(s) Summary
Config migration v12 & legacy key pruning
installer/steps/config_migration.py
Config version bumped to 12. Migration writes a one-time <config>.json.bak.v11 snapshot for pre-v12 configs, prunes legacy model-routing keys (model, skills, agents, extendedContext, extendedContextOverrides), ensures specWorkflow exists, and seeds specWorkflow.modelSwitch when missing.

Deprecated Artifact Cleanup

Layer / File(s) Summary
Deprecated skill cleanup in claude_files.py
installer/steps/claude_files.py, installer/tests/unit/steps/test_claude_files.py
Adds _cleanup_deprecated_pilot_skills_in_home to remove exact-named deprecated Pilot skill directories and agent filenames under ~/.claude; called before manifest seeding and covered by idempotency tests.

Installation & Messaging Infrastructure

Layer / File(s) Summary
Outcome tracking infrastructure & messaging
installer/steps/dependencies.py, installer/tests/unit/steps/test_dependencies.py
Introduces a thread-local outcome sidechannel and recording helpers; install/cleanup functions record INSTALLED/UPDATED/UNCHANGED/REMOVED and UI now reports accurate outcomes rather than always “installed”.
Plugin management refactoring & context-mode removal
installer/steps/dependencies.py
Refactors _install_or_update_plugin() with defensive JSON parsing and idempotent enabling; adds remove_legacy_context_mode() to uninstall legacy context-mode plugin/marketplace and remove hook artifacts, run sequentially after Claude Code install.
Uninstall script legacy cleanup
uninstall.sh
Adds arrays for extra/legacy plugin IDs, marketplaces, and hook files plus uninstall_extra_plugins() to remove them via claude CLI; integrated into uninstall order after LSP plugin removal.
Install script error handling reformatting
install.sh
Refactors fetch failure handling in install_uv() to explicit multi-line curl/wget branches; removes an upstreams comment; behavior unchanged.

Documentation & Marketing Content

Layer / File(s) Summary
README updates
README.md
Removes Security Scanner references; updates Code Intelligence/Token Optimization wording; adds “Reset & Refresh” procedure; updates installer dependency list to Semble/RTK/CodeGraph and browser automation tooling; emphasizes Model Switching in Console Settings and updates FAQ.
Feature documentation updates
docs/docusaurus/docs/features/model-routing.md, docs/docusaurus/docs/features/hooks.md, docs/docusaurus/docs/features/rules.md, docs/docusaurus/docs/features/context-optimization.md
Model-routing rewritten to single /model switch with /spec-driven pause/resume flow; credential-scanner hook entries removed from hooks docs; rules reduced (context-mode removed); context-optimization adds Semble code search and routing hooks.
MCP & tools documentation
docs/docusaurus/docs/features/mcp-servers.md, docs/docusaurus/docs/features/open-source-tools.md
MCP servers doc now references context7 and chrome-devtools-mcp, blocks direct curl/wget/WebFetch and points to web-fetch/web-search MCP servers; open-source-tools removes context-mode and repurposes better-sqlite3/Turndown descriptions.
Console & installation documentation
docs/docusaurus/docs/features/console.md, docs/docusaurus/docs/getting-started/installation.md, docs/docusaurus/docs/features/permission-modes.md
Console Settings/docs show Spec Workflow toggles including Model Switching; ~/.pilot/config.json example gains modelSwitch; Getting Started adds Reset & Refresh; permission-modes sidebar position updated.
Sidebar navigation
docs/docusaurus/sidebars.ts
Removes permission-modes from Getting Started and replaces features/security with features/permission-modes in Features.
Marketing site content
docs/site/src/components/ConsoleSection.tsx, docs/site/src/components/DeepDiveSection.tsx, docs/site/src/components/FAQSection.tsx, docs/site/src/components/WhatsInside.tsx
Updates Usage/Settings slides, DeepDive hooks/context/model-switching copy, adds Reset & Refresh FAQ, and updates feature cards to reflect Hooks & Quality and Model Switching text.

Tests

Layer / File(s) Summary
Config migration tests
installer/tests/unit/steps/test_config_migration.py
Refactors many end-to-end migration tests to unit tests of _migration_v*; adds comprehensive v12 tests validating pruning, seeding, one-time backup write, backup idempotency, and idempotent migration behavior.
Deprecated cleanup tests
installer/tests/unit/steps/test_claude_files.py
Adds tests for deprecated skills cleanup and idempotency; preserves other test assertions with formatting updates.
Dependencies & plugin tests
installer/tests/unit/steps/test_dependencies.py
Replaces context-mode install tests with remove_legacy_context_mode coverage; enhances Chrome DevTools plugin tests for enablement behavior and malformed JSON resilience; updates mocks accordingly.
Other test updates
installer/tests/unit/steps/test_prerequisites.py, installer/tests/unit/test_drift_check.py, installer/tests/unit/test_install_sh.py
Minor formatting and assertion pattern adjustments to align with code changes; no functional test logic changes.

Estimated code review effort:
🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

Suggested labels

released

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 74.47% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main changes: model switching for /spec workflow plus legacy systems cleanup for v9.0.0, directly corresponding to the major breaking changes and feature additions in the changeset.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch dev

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@claude

claude Bot commented May 22, 2026

Copy link
Copy Markdown

Claude finished @maxritter's task in 2m 55s —— View job


Code Review: v9.0.0 — Model switching for /spec + legacy-systems cleanup

Tasks

  • Gather context and read PR description
  • Review installer changes (config migration, dependencies, claude_files)
  • Review pilot hooks (new spec hooks, cleaned-up hooks)
  • Review documentation changes
  • Review install.sh / uninstall.sh
  • Review tests coverage
  • Post final review

Overview

This is a well-structured major release. The commit split (additive feature first, breaking cleanup second) makes the changelog genuinely readable, and the underlying sentinel-based handoff design for model switching is clean. The config-migration backup, the outcome-sidechannel in the installer, and the deprecation-allowlist approach for skills cleanup are all solid patterns. Below are the issues I found, roughly ordered by severity.


Bug — "plugin" vs "plugins" in legacy context-mode cleanup

_legacy_context_mode_uninstall_plugin and _legacy_context_mode_remove_marketplace both call the Claude CLI using the singular subcommand ("plugin"):

# installer/steps/dependencies.py — _legacy_context_mode_uninstall_plugin
["claude", "plugin", "list", "--json"]
["claude", "plugin", "uninstall", _LEGACY_CONTEXT_MODE_PLUGIN_ID, "-y"]

# _legacy_context_mode_remove_marketplace
["claude", "plugin", "marketplace", "list", "--json"]
["claude", "plugin", "marketplace", "remove", _LEGACY_CONTEXT_MODE_MARKETPLACE]

Every other Claude CLI invocation in the same file and in uninstall.sh uses the plural form ("plugins"):

# _install_or_update_plugin — existing, working code
["claude", "plugins", "list", "--json"]
["claude", "plugins", "update", plugin_id]
["claude", "plugins", "install", plugin_id]

# _ensure_plugin_enabled
["claude", "plugins", "enable", plugin_id]

If the CLI only accepts plugins (plural), the pre-flight list --json calls will return non-zero, both helper functions will return False, remove_legacy_context_mode will log no output (outcome unchanged), and the old plugin will silently persist. The tests mock subprocess.run at the module level so they can't catch this.

Fix this →


Important — Non-atomic write to ~/.claude/settings.json

_legacy_context_mode_remove_orphan_hook writes the updated settings file directly:

# installer/steps/dependencies.py — _legacy_context_mode_remove_orphan_hook
settings_path.write_text(json.dumps(data, indent=2) + "\n")

A crash or power-loss mid-write will truncate or corrupt ~/.claude/settings.json, which can break the Claude CLI for the user. The module already has _write_atomic (temp-file + os.rename) for exactly this kind of update.

Fix this →


Minor — Misleading warning in spec_mode_guard.py

When the statusline cache is unavailable the warning tells the user to run /clear before /model:

# pilot/hooks/spec_mode_guard.py
"if you are on Sonnet, run '/clear' then '/model opus[1m]' before planning."

/model works without a session clear — recommending /clear here mirrors Path B (clean start) from the handoff docs, but in this context (the user hasn't started planning yet) the clear is unnecessary and could discard in-progress work.

Fix this →


Observations (non-blocking)

spec_handoff_resume.py sentinel ordering is intentional but worth noting. The sentinel is unlinked before the age check — meaning it is always consumed on the first post-handoff prompt even when stale. This is the right call (prevents the hook from triggering on a second prompt if the user ignored the first), but it's a non-obvious ordering for a future reader.

_migration_v12 docstring mentions the caller writes the backup — true, but the backup file is named config.json.bak.v11 (implying it's the v11 state). Naming is accurate and the PR description explains it well; a one-line note in the docstring referencing the exact file name would help future maintainers.

Test refactoring for v1–v11 migration tests is a real improvement. Moving from round-tripping through migrate_model_config with temp files to calling _migration_vN in-memory makes each test intent-clear and 10× faster. The new test_v12_* tests follow this same pattern correctly.

_DEPRECATED_PILOT_SKILL_NAMES allowlist in claude_files.py is exactly the right safety mechanism for shared directories where user files may live alongside Pilot-managed files. Explicit names only, no wildcards — safe.

Outcome sidechannel (_record_outcome / _take_outcome) avoids turning every install_* function into a two-tuple return, keeping the existing mock-able bool contract intact. The explicit _take_outcome() drain in _run_install_silent before each call is necessary and is present.


Summary

One real bug (singular vs. plural CLI subcommand for the legacy cleanup that will cause silent no-ops), one data-safety issue (non-atomic settings write), and one misleading UX string. The core model-switching architecture — sentinel files, stop-guard coordination, handoff-resume hook, Opus guard — is solid and well-tested.

@maxritter maxritter self-assigned this May 22, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
installer/tests/unit/steps/test_config_migration.py (1)

147-167: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Test intent and assertions are mismatched for “subscription-aware” fresh install.

This test’s docstring describes a Max-plan regression on default model routing, but the assertions only verify file creation and _configVersion. It doesn’t assert any subscription-dependent outcome, so it won’t catch the regression it describes.

Please either (a) assert a concrete max-specific postcondition, or (b) rename/re-scope the test to reflect what it actually validates.

As per coding guidelines, “Review test code briefly. Focus on coverage and proper mocking.”

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/docusaurus/docs/features/console.md`:
- Line 106: Update the Settings tab description to remove the stale "Security"
section reference: modify the sentence that currently reads "a single scrollable
page with three stacked sections: **Spec Workflow**, **Console**, **Security**"
to reflect the current UI (e.g., "a single scrollable page with two stacked
sections: **Spec Workflow** and **Console**"), and ensure the following
sentences about toggle preferences saving to `~/.pilot/config.json` and the
**Console → Worker Port** saving to `~/.pilot/memory/settings.json` remain
accurate and unchanged; update any nearby wording that mentions restarting Pilot
if necessary so the flow no longer references a Security tab.

In `@docs/docusaurus/docs/getting-started/installation.md`:
- Around line 80-83: The bash snippet contains an invalid token "> /logout"
which is interpreted as shell redirection; update the code block to remove the
leading ">" so it reads simply "/logout" (or move the "/logout" instruction out
of the bash fenced block into a plain text step). Modify the block that
currently contains "> /logout" to either show "/logout" alone inside the fenced
bash snippet or present the logout instruction as a regular list item so
copy-paste runs correctly.

In `@docs/site/src/components/ConsoleSection.tsx`:
- Around line 64-65: The Settings copy in ConsoleSection.tsx still mentions
"credential scanner" even though that feature and the Console Security tab were
removed; update the descriptive strings (the desc prop and, if present, the alt
text) in the ConsoleSection component to remove the phrase "credential scanner"
and any related wording so the UI text reflects the current feature set (keep
the rest of the description about spec workflow toggles, reviewer configuration,
and model note intact).

In `@installer/steps/dependencies.py`:
- Around line 811-870: The subprocess invocations in
_legacy_context_mode_uninstall_plugin and
_legacy_context_mode_remove_marketplace use the wrong subcommand "plugin"
(singular) causing commands to fail; update each subprocess.run call in those
functions to use "plugins" (plural) for the CLI subcommand (replace ["claude",
"plugin", ...] with ["claude", "plugins", ...] in the
list/marketplace/uninstall/remove calls so the calls in
_legacy_context_mode_uninstall_plugin and
_legacy_context_mode_remove_marketplace match the working usage elsewhere).

In `@installer/tests/unit/steps/test_config_migration.py`:
- Around line 1110-1111: The test docstrings and assertions for
migrate_model_config full-chain tests are inconsistent about what the .bak.v11
backup should contain (pre-v12 vs original input); pick one contract (recommend:
.bak.v11 captures the state after v9/v10 but before v12 pruning) and make
tests/docstrings consistent: update the docstring that currently says ".bak.v11
captures pre-v12 state" to match the chosen contract if needed, and change
assertions that expect original-input values (e.g., expecting "opus" or "[1m]")
to instead assert the post-v9/post-v10 form that should appear in the .bak.v11
backup; apply the same alignment to other affected tests referenced (around the
blocks near lines 1139-1144, 1266-1267, 1290-1295) and ensure references to
.bak.v11, migrate_model_config, and the specific test functions reflect the
chosen snapshot contract.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: b456be5d-ff74-4311-bdef-bf49f709902a

📥 Commits

Reviewing files that changed from the base of the PR and between 37afe8f and 63e46e9.

⛔ Files ignored due to path filters (102)
  • console/src/services/worker/http/routes/ExtensionRoutes.ts is excluded by !console/**
  • console/src/services/worker/http/routes/SettingsRoutes.ts is excluded by !console/**
  • console/src/services/worker/http/routes/ToolSavingsRoutes.ts is excluded by !console/**
  • console/src/ui/viewer/App.tsx is excluded by !console/**
  • console/src/ui/viewer/components/ActivationModal.tsx is excluded by !console/**
  • console/src/ui/viewer/components/LicenseGate.tsx is excluded by !console/**
  • console/src/ui/viewer/hooks/useLegacySettingsRedirect.ts is excluded by !console/**
  • console/src/ui/viewer/hooks/useSettings.ts is excluded by !console/**
  • console/src/ui/viewer/hooks/useToolSavings.ts is excluded by !console/**
  • console/src/ui/viewer/views/Settings/ModelSelect.tsx is excluded by !console/**
  • console/src/ui/viewer/views/Settings/index.tsx is excluded by !console/**
  • console/src/ui/viewer/views/Settings/sections/ConsoleSection.tsx is excluded by !console/**
  • console/src/ui/viewer/views/Settings/sections/ModelsSection.tsx is excluded by !console/**
  • console/src/ui/viewer/views/Settings/sections/SecuritySection.tsx is excluded by !console/**
  • console/src/ui/viewer/views/Settings/sections/SpecWorkflowSection.tsx is excluded by !console/**
  • console/src/ui/viewer/views/Settings/sections/_shared.tsx is excluded by !console/**
  • console/src/ui/viewer/views/Usage/ModelRoutingInfo.tsx is excluded by !console/**
  • console/src/ui/viewer/views/Usage/index.tsx is excluded by !console/**
  • console/tests/hooks/useSettings-save-error.test.tsx is excluded by !console/**
  • console/tests/hooks/useSettings.test.ts is excluded by !console/**
  • console/tests/settings-routes.test.ts is excluded by !console/**
  • console/tests/ui/legacy-settings-redirect.test.tsx is excluded by !console/**
  • console/tests/ui/model-routing-info.test.ts is excluded by !console/**
  • console/tests/ui/usage-view.test.ts is excluded by !console/**
  • console/tests/worker/extension-routes.test.ts is excluded by !console/**
  • launcher/banner.py is excluded by !launcher/**
  • launcher/build.py is excluded by !launcher/**
  • launcher/cli.py is excluded by !launcher/**
  • launcher/model_config.py is excluded by !launcher/**
  • launcher/notice.py is excluded by !launcher/**
  • launcher/settings_injector.py is excluded by !launcher/**
  • launcher/statusline/formatter.py is excluded by !launcher/**
  • launcher/tests/unit/statusline/test_formatter.py is excluded by !launcher/**
  • launcher/tests/unit/test_ansi_safety.py is excluded by !launcher/**
  • launcher/tests/unit/test_customize.py is excluded by !launcher/**
  • launcher/tests/unit/test_model_config.py is excluded by !launcher/**
  • launcher/tests/unit/test_notice.py is excluded by !launcher/**
  • launcher/tests/unit/test_settings_injector.py is excluded by !launcher/**
  • launcher/wrapper.py is excluded by !launcher/**
  • pilot/claude.json is excluded by !pilot/**
  • pilot/hooks/_checkers/tdd.py is excluded by !pilot/**
  • pilot/hooks/_lib/allow_tags.py is excluded by !pilot/**
  • pilot/hooks/_lib/secret_scanner.py is excluded by !pilot/**
  • pilot/hooks/_lib/util.py is excluded by !pilot/**
  • pilot/hooks/context_monitor.py is excluded by !pilot/**
  • pilot/hooks/credential_scanner.py is excluded by !pilot/**
  • pilot/hooks/hooks.json is excluded by !pilot/**
  • pilot/hooks/session_clear.py is excluded by !pilot/**
  • pilot/hooks/spec_handoff_resume.py is excluded by !pilot/**
  • pilot/hooks/spec_mode_guard.py is excluded by !pilot/**
  • pilot/hooks/spec_stop_guard.py is excluded by !pilot/**
  • pilot/hooks/tests/test__util.py is excluded by !pilot/**
  • pilot/hooks/tests/test_allow_tags.py is excluded by !pilot/**
  • pilot/hooks/tests/test_context_monitor.py is excluded by !pilot/**
  • pilot/hooks/tests/test_credential_scanner.py is excluded by !pilot/**
  • pilot/hooks/tests/test_secret_scanner.py is excluded by !pilot/**
  • pilot/hooks/tests/test_spec_handoff_resume.py is excluded by !pilot/**
  • pilot/hooks/tests/test_spec_mode_guard.py is excluded by !pilot/**
  • pilot/hooks/tests/test_spec_stop_guard.py is excluded by !pilot/**
  • pilot/hooks/tests/test_tdd_enforcer.py is excluded by !pilot/**
  • pilot/hooks/tests/test_tool_redirect.py is excluded by !pilot/**
  • pilot/hooks/tool_redirect.py is excluded by !pilot/**
  • pilot/rules/browser-automation.md is excluded by !pilot/**
  • pilot/rules/context-mode.md is excluded by !pilot/**
  • pilot/rules/development-practices.md is excluded by !pilot/**
  • pilot/rules/security-credentials.md is excluded by !pilot/**
  • pilot/scripts/mcp-server.cjs is excluded by !pilot/**
  • pilot/scripts/worker-service.cjs is excluded by !pilot/**
  • pilot/settings.json is excluded by !pilot/**
  • pilot/skills/benchmark/orchestrator.md is excluded by !pilot/**
  • pilot/skills/benchmark/scripts/isolation.py is excluded by !pilot/**
  • pilot/skills/benchmark/scripts/runner.py is excluded by !pilot/**
  • pilot/skills/benchmark/scripts/utils.py is excluded by !pilot/**
  • pilot/skills/benchmark/steps/04-execute.md is excluded by !pilot/**
  • pilot/skills/benchmark/tests/test_aggregate_benchmark.py is excluded by !pilot/**
  • pilot/skills/benchmark/tests/test_isolation.py is excluded by !pilot/**
  • pilot/skills/benchmark/tests/test_runner.py is excluded by !pilot/**
  • pilot/skills/benchmark/tests/test_utils.py is excluded by !pilot/**
  • pilot/skills/create-skill/orchestrator.md is excluded by !pilot/**
  • pilot/skills/fix/orchestrator.md is excluded by !pilot/**
  • pilot/skills/fix/steps/06-finalise.md is excluded by !pilot/**
  • pilot/skills/prd/orchestrator.md is excluded by !pilot/**
  • pilot/skills/setup-rules/orchestrator.md is excluded by !pilot/**
  • pilot/skills/spec-bugfix-plan/orchestrator.md is excluded by !pilot/**
  • pilot/skills/spec-bugfix-plan/steps/00-setup-and-red-flags.md is excluded by !pilot/**
  • pilot/skills/spec-bugfix-plan/steps/06-approval.md is excluded by !pilot/**
  • pilot/skills/spec-bugfix-verify/orchestrator.md is excluded by !pilot/**
  • pilot/skills/spec-implement/orchestrator.md is excluded by !pilot/**
  • pilot/skills/spec-plan/orchestrator.md is excluded by !pilot/**
  • pilot/skills/spec-plan/steps/00-setup-and-questions.md is excluded by !pilot/**
  • pilot/skills/spec-plan/steps/10-plan-verification.md is excluded by !pilot/**
  • pilot/skills/spec-plan/steps/12-approval.md is excluded by !pilot/**
  • pilot/skills/spec-verify/orchestrator.md is excluded by !pilot/**
  • pilot/skills/spec-verify/steps/01-launch-review.md is excluded by !pilot/**
  • pilot/skills/spec-verify/steps/03-collect-results.md is excluded by !pilot/**
  • pilot/skills/spec-verify/steps/07-e2e-and-final-regression.md is excluded by !pilot/**
  • pilot/skills/spec/orchestrator.md is excluded by !pilot/**
  • pilot/ui/ExtensionsView.js is excluded by !pilot/**
  • pilot/ui/index4.js is excluded by !pilot/**
  • pilot/ui/index5.js is excluded by !pilot/**
  • pilot/ui/viewer-bundle.js is excluded by !pilot/**
  • pilot/ui/viewer.css is excluded by !pilot/**
📒 Files selected for processing (27)
  • README.md
  • docs/docusaurus/docs/features/console.md
  • docs/docusaurus/docs/features/context-optimization.md
  • docs/docusaurus/docs/features/hooks.md
  • docs/docusaurus/docs/features/mcp-servers.md
  • docs/docusaurus/docs/features/model-routing.md
  • docs/docusaurus/docs/features/open-source-tools.md
  • docs/docusaurus/docs/features/permission-modes.md
  • docs/docusaurus/docs/features/rules.md
  • docs/docusaurus/docs/features/security.md
  • docs/docusaurus/docs/getting-started/installation.md
  • docs/docusaurus/sidebars.ts
  • docs/site/src/components/ConsoleSection.tsx
  • docs/site/src/components/DeepDiveSection.tsx
  • docs/site/src/components/FAQSection.tsx
  • docs/site/src/components/WhatsInside.tsx
  • install.sh
  • installer/steps/claude_files.py
  • installer/steps/config_migration.py
  • installer/steps/dependencies.py
  • installer/tests/unit/steps/test_claude_files.py
  • installer/tests/unit/steps/test_config_migration.py
  • installer/tests/unit/steps/test_dependencies.py
  • installer/tests/unit/steps/test_prerequisites.py
  • installer/tests/unit/test_drift_check.py
  • installer/tests/unit/test_install_sh.py
  • uninstall.sh
💤 Files with no reviewable changes (1)
  • docs/docusaurus/docs/features/security.md

Comment thread docs/docusaurus/docs/features/console.md Outdated
Comment thread docs/docusaurus/docs/getting-started/installation.md
Comment thread docs/site/src/components/ConsoleSection.tsx Outdated
Comment thread installer/steps/dependencies.py
Comment on lines +1110 to +1111
"""Full migrate_model_config from v8 runs v9 (sonnet for pro) then v12 (prune model keys).
The v9 effect is captured in the .bak.v11 backup; the on-disk config has the keys pruned."""

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Backup-state expectations are internally inconsistent across full-chain tests.

These tests/documentation say the .bak.v11 captures pre-v12 state, but assertions expect pre-v9/pre-v10 values (e.g., expecting opus/[1m] forms). This ambiguity can lock in the wrong migration contract.

Please align docstrings/comments and assertions to one contract (pre-v12 snapshot vs original input snapshot) consistently.

Also applies to: 1139-1144, 1266-1267, 1290-1295

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@installer/tests/unit/steps/test_config_migration.py` around lines 1110 -
1111, The test docstrings and assertions for migrate_model_config full-chain
tests are inconsistent about what the .bak.v11 backup should contain (pre-v12 vs
original input); pick one contract (recommend: .bak.v11 captures the state after
v9/v10 but before v12 pruning) and make tests/docstrings consistent: update the
docstring that currently says ".bak.v11 captures pre-v12 state" to match the
chosen contract if needed, and change assertions that expect original-input
values (e.g., expecting "opus" or "[1m]") to instead assert the post-v9/post-v10
form that should appear in the .bak.v11 backup; apply the same alignment to
other affected tests referenced (around the blocks near lines 1139-1144,
1266-1267, 1290-1295) and ensure references to .bak.v11, migrate_model_config,
and the specific test functions reflect the chosen snapshot contract.

- `docs/features/console.md`: Settings tab now correctly describes two
  stacked sections (Spec Workflow and Console) — the removed Security
  section was still mentioned.
- `docs/getting-started/installation.md` + `README.md`: drop the leading
  `>` before `/logout` inside the Reset & Refresh bash fence. The `>`
  was meant as a Claude Code prompt marker but inside a fenced bash
  block it parses as shell redirection, so copy-paste would silently
  truncate the file to empty.
- `docs/site/src/components/ConsoleSection.tsx`: remove "credential
  scanner" and "security toggles" copy from the Settings card; both
  features have been removed.
- `installer/steps/dependencies.py`: standardize `claude plugin` to
  `claude plugins` in the legacy-context-mode cleanup family.
  `claude --help` shows `claude plugin|plugins` as accepted aliases —
  both forms behave identically in CLI v2.1.148 — but the inconsistency
  with the install/update path (which already uses the plural form)
  made the legacy cleanup look broken at a glance. Tests updated to
  match.
@claude

claude Bot commented May 22, 2026

Copy link
Copy Markdown

Claude finished @maxritter's task in 2m 21s —— View job


Incremental Code Review — push 5afbf311

Tasks

  • Gather context and prior review findings
  • Read changed files (README.md, console.md, installation.md, ConsoleSection.tsx, dependencies.py, test_dependencies.py)
  • Check which prior issues are resolved
  • Check for new issues in the push
  • Post final review

✅ Resolved by this push

pluginplugins consistency in legacy cleanup (installer/steps/dependencies.py)**
All four subprocess.run calls in _legacy_context_mode_uninstall_plugin and _legacy_context_mode_remove_marketplace now use "plugins" (plural), matching the install/update path. The commit message usefully notes that both forms are accepted CLI aliases in v2.1.148 — so this was a consistency fix rather than a functional bug. Tests updated correctly in test_dependencies.py to match.

Stale "Security" section removed from console.md (CodeRabbit finding)
Settings tab description now correctly reads "two stacked sections: Spec Workflow and Console".

> /logout shell-redirection hazard fixed in both README.md and installation.md (CodeRabbit finding)
The leading > has been dropped from the Reset & Refresh bash snippet; /logout now appears as a bare token that won't truncate the file on copy-paste.

"credential scanner" copy removed from ConsoleSection.tsx (CodeRabbit finding)
alt and desc strings both updated — "security toggles" and "credential scanner" are gone, replaced with accurate "spec workflow and console port" copy.


Still open

  • Non-atomic write to ~/.claude/settings.json_legacy_context_mode_remove_orphan_hook at installer/steps/dependencies.py:932 still calls settings_path.write_text(...) directly. _write_atomic exists in config_migration.py but is not imported or used here. A crash mid-write will corrupt the file. Fix this →
  • Misleading /clear in spec_mode_guard.py — line 148 still reads "run '/clear' then '/model opus[1m]' before planning". At that point in the flow the user hasn't started a plan, so recommending /clear could discard in-progress work unnecessarily. Fix this →

No new issues introduced

All six changed files in this push contain mechanical, targeted fixes. No regressions or new problems found.

@maxritter maxritter merged commit 6de06c5 into main May 22, 2026
15 of 16 checks passed
@maxritter maxritter deleted the dev branch May 22, 2026 09:25

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 129 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="pilot/claude.json">

<violation number="1" location="pilot/claude.json:4">
P2: `autoConnectIde` and `autoInstallIdeExtension` are Claude Code `settings.json` settings, not `~/.claude.json` preferences. Placing them in `pilot/claude.json` (which merges into `~/.claude.json`) means Claude Code may not read them or may log unrecognized-key warnings.

Move these to a `settings.json` customization source that targets `~/.claude/settings.json`.</violation>
</file>

Note: This PR contains a large number of files. cubic only reviews up to 100 files per PR, so some files may not have been reviewed. cubic prioritizes the most important files to review.
On a pro plan you can use ultrareview for larger PRs.

Fix all with cubic | Re-trigger cubic

Comment thread pilot/claude.json
{
"showExpandedTodos": true,
"autoUpdates": true,
"autoConnectIde": true,

@cubic-dev-ai cubic-dev-ai Bot May 26, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: autoConnectIde and autoInstallIdeExtension are Claude Code settings.json settings, not ~/.claude.json preferences. Placing them in pilot/claude.json (which merges into ~/.claude.json) means Claude Code may not read them or may log unrecognized-key warnings.

Move these to a settings.json customization source that targets ~/.claude/settings.json.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At pilot/claude.json, line 4:

<comment>`autoConnectIde` and `autoInstallIdeExtension` are Claude Code `settings.json` settings, not `~/.claude.json` preferences. Placing them in `pilot/claude.json` (which merges into `~/.claude.json`) means Claude Code may not read them or may log unrecognized-key warnings.

Move these to a `settings.json` customization source that targets `~/.claude/settings.json`.</comment>

<file context>
@@ -1,6 +1,8 @@
 {
   "showExpandedTodos": true,
   "autoUpdates": true,
+  "autoConnectIde": true,
+  "autoInstallIdeExtension": true,
   "lspRecommendationDisabled": true,
</file context>
Fix with Cubic

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant