Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 10 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ High-level flow:

1. You send a prompt through Switchboard.
2. Switchboard classifies the turn and selects a route label.
3. Switchboard launches or resumes Claude with matching model and effort settings.
3. Switchboard launches or resumes Claude with matching model and effort settings for that launch.
4. Route context, session state, and hook evidence are recorded for explainability, replay, and governance.

## What It Is Not
Expand All @@ -63,13 +63,21 @@ See [SECURITY.md](SECURITY.md) for details on the vulnerability reporting proces
| Command | What It Does | Use It When |
| --- | --- | --- |
| `switchboard "your prompt"` | Routes a single prompt, chooses target/effort, then launches or resumes Claude for that turn. | You want normal prompt-driven usage with routing applied automatically. |
| `switchboard --interactive` | Starts an interactive Claude session through Switchboard with route-aware session handling. | You want a live back-and-forth session instead of one-shot prompts. |
| `switchboard --interactive` | Starts an interactive Claude session through Switchboard with route-aware session handling. The launched Claude session keeps the selected model/effort; it does not auto-switch models mid-session. | You want a live back-and-forth session instead of one-shot prompts. |
| `switchboard explain` | Shows the latest routing decision, reasoning signals, and correlated evidence for a thread. | You want to audit why a route was chosen or debug routing behavior. |
| `switchboard advise --surface openai-codex "your prompt"` | Returns an advisory routing recommendation for a selected surface without taking over execution. | You want a cross-surface recommendation or policy check before running a turn. |
| `switchboard probe continuity` | Runs a continuity probe for prompt-driven turns and reports whether session continuity checks pass. | You want to verify non-interactive continuity behavior after changes. |
| `switchboard probe continuity-interactive` | Runs the interactive continuity probe and verifies resume/session behavior across turns. | You want to validate interactive continuity and related checks. |
| `npm test` | Runs the full automated test suite for adapters, router, workflow, and CLI behavior. | You changed routing/workflow/docs and want a full regression check. |

### Interactive Mode Clarification

`switchboard --interactive` selects model/effort at launch, then enters Claude interactive mode with that selection.

Inside that active Claude session, Switchboard does not automatically switch to a different model on later user messages.

If you want per-turn re-routing and potential target/model changes, run prompts through Switchboard as separate turns (for example: `switchboard "..."` each turn) rather than staying in one interactive session.

### Typical First Run

1. Send one routed prompt:
Expand Down
44 changes: 44 additions & 0 deletions docs/decision-log.md
Original file line number Diff line number Diff line change
Expand Up @@ -288,3 +288,47 @@ Consequences:
Follow-up:
- Next review milestone: Milestone 5 second-surface proof conditional gate (review stability, integration readiness, contract reusability).
- Linked artifacts (logs, fixtures, docs, PRs): src/router/outcome-constants.js, src/switchboard/attribution-store.js, src/switchboard/workflow.js, src/switchboard/cli.js, test/attribution.test.js, docs/contracts/router-contracts.md, docs/REPLAY-GUIDE.md, docs/product/ROUTER-PHASE-PLAN.md

## Open Product Risk: Per-Turn Routing UX Friction

Decision ID: DEC-2026-05-13-per-turn-routing-ux-friction
Related deferred item: MVP-PRD.md Assumption B (Claude Continuity) falsification criteria; PRODUCT-PRD.md Section 17.1 deferred items
Status: committed
Date: 2026-05-13
Owners: team

Context:
- Manual testing and review of interactive mode surfaced a gap between technical continuity and UX continuity. The continuity assumption (Assumption B in MVP-PRD.md) was verified against session-id persistence and chat history carry-over, but not against the workflow interaction pattern the user must follow to obtain per-turn routing.
- In practice, per-turn routing (the only path that gets automatic model/effort re-selection each turn) requires the user to: exit Claude, return to the terminal, invoke `switchboard "..."`, wait for routing, and re-enter Claude for each routed turn (typically by resuming the same Claude session id). This is materially different from the normal Claude Code usage pattern, where the user stays inside one interactive session.
- Interactive mode (`switchboard --interactive`) avoids this friction but fixes the model/effort at launch with no re-routing once inside Claude.
- There is currently no path that provides both: per-turn automatic re-routing and a natural stay-in-session experience. This trading of one cognitive cost (model selection) for another (repeated session cycling) risks limiting practical adoption among the stated target user: high-context engineers already comfortable with Claude Code.

Options considered:
- Option A: accept the current per-turn pattern as a known limitation and focus adoption documentation on making the friction explicit rather than eliminating it.
- Option B: invest in a hook-based advisory injection path that surfaces the next-turn route recommendation inside the running Claude session, so the user at least sees what routing would suggest without having to exit and re-enter.
- Option C: defer until Claude exposes a supported in-session model-switching API, at which point routing authority can move inside the session rather than at the launch boundary.
- Option D: redesign the primary usage pattern around `switchboard --interactive` with acceptance that model/effort is set once per session, and treat per-turn rerouting as an advanced or explicit-override use case rather than the default experience.

Tradeoffs:
- Option A: honest and low effort, but does not resolve the adoption risk and may position the tool as too operationally disruptive for daily use.
- Option B: provides routing visibility without forcing session cycling; feasible with the existing hook bridge advisory injection; does not require a new Claude API; does not actually change the model mid-session, but closes some of the perception gap. Requires clear UX language so users understand it is advisory, not authoritative.
- Option C: no product work needed now; entirely contingent on a vendor capability that does not currently exist. Appropriate as a deferred path but not a near-term strategy.
- Option D: reduces the scope of the routing promise to session-start only; aligns better with the natural Claude Code usage pattern; may feel like a weaker product but is more honest about what the current architecture delivers.

Verification signal:
- Expected signal: evidence that real users (or the product owner in daily use) find the per-turn cycling acceptable or unacceptable in practice.
- Evidence observed: manual testing and reasoning session on 2026-05-13 identified the friction clearly; no live adoption data yet as the product is pre-broad-release.

Decision:
- Chosen option: Option B — invest in hook-based advisory injection that surfaces the routing recommendation inside the running Claude session via `UserPromptSubmit` context injection. This is the best available path given current constraints: it requires no new Claude API, builds on existing hook infrastructure, and closes the perception gap without forcing session cycling.
- Scope of commitment: pursue Option B as the near-term direction. Separately, survey alternative client surfaces (e.g. Cursor, GitHub Copilot Chat, Gemini CLI, OpenClaw) to identify whether any expose an integration point that allows genuine per-turn routing authority within a running session, which would supersede the advisory approach.
- What remains intentionally deferred: Option C (true in-session model switching) remains deferred pending a supported Claude API. Option D (reframing the product promise) is kept as a fallback if client exploration yields no better interface. The client exploration work should inform a follow-up decision before broad adoption promotion.

Consequences:
- Near-term implementation impact: hook bridge advisory injection path needs to be hardened so that each `UserPromptSubmit` event surfaces a clear, low-noise routing recommendation the user can act on or ignore. This is distinct from current advisory context injection which is primarily for correlated logging.
- Test and replay impact: new advisory injection behavior should be covered by hook bridge tests to ensure recommendation accuracy and noise level are acceptable.
- Migration impact: low; existing hook correlation and logging behavior is preserved. Advisory injection is additive.

Follow-up:
- Next review milestone: (1) implement and live-verify improved advisory injection; (2) complete a client surface survey covering at minimum Cursor, GitHub Copilot Chat, Gemini CLI, and one gateway-backed path; (3) revisit Option C and Option D based on survey findings before broad adoption promotion.
- Linked artifacts (logs, fixtures, docs, PRs): docs/product/MVP-PRD.md (Assumption B, Section 8 MVP defer list), docs/product/PRODUCT-PRD.md (Section 17.1 deferred items updated 2026-05-13), README.md (Interactive Mode Clarification section added 2026-05-13), src/switchboard/claude-hook-bridge.js (advisory injection comment)
2 changes: 1 addition & 1 deletion docs/product/PRODUCT-PRD.md
Original file line number Diff line number Diff line change
Expand Up @@ -278,7 +278,7 @@ These items were explicitly deferred from the Claude Code MVP. They are the firs

**`deep reasoning` as a standalone target**: Only worth adding if a vendor exposes a meaningful distinction from `balanced` or `best coder` routing. No current Claude evidence supports a useful distinction.

**In-session automatic model switching**: Hooks cannot change the active model inside a running Claude session through any supported mechanism. Defer unless Claude exposes a supported hook output for this.
**In-session automatic model switching**: Hooks cannot change the active model inside a running Claude session through any supported mechanism. Near-term direction (committed 2026-05-13, DEC-2026-05-13-per-turn-routing-ux-friction): pursue hook-based advisory injection that surfaces the routing recommendation inside the running session via `UserPromptSubmit`, so users can act on it without being forced to cycle sessions. True in-session switching remains deferred until Claude exposes a supported hook output. A client surface survey (Cursor, GitHub Copilot Chat, Gemini CLI, gateway-backed paths) is also planned to identify whether any alternative surface provides better per-turn routing authority.

**Fully interactive Claude shell parity**: `switchboard --interactive` is implemented and live-verified for basic continuity. It is not yet considered fully supported. See Section 17.2 for the validation criteria before removing the experimental label.

Expand Down