Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,38 @@ env = { SCRAPECLI_API_KEY = "env:SCRAPECLI_TEAM_B_KEY" }

Retry rules, auth command names, and usage endpoints for SearchCLI and ScrapeCLI ship in `src/clifwrap/providers.toml`. Your config usually only needs account entries.

### Interactive agent CLIs

Some coding-agent CLIs ship a full-screen TUI plus a headless print mode and authenticate with a per-account API key env var. The catalog names `agent` and `agentcli` are templates — wrap whatever binary resolves on your PATH and set the key env var the tool actually reads.

```bash
clifwrap init
clifwrap account add agent primary --env-ref AGENTCLI_API_KEY=AGENTCLI_API_KEY_PRIMARY
clifwrap account add agent backup --env-ref AGENTCLI_API_KEY=AGENTCLI_API_KEY_BACKUP
clifwrap install agent
# If another tool already owns `agent` on PATH, wrap the alternate name instead:
# clifwrap account add agentcli primary --env-ref AGENTCLI_API_KEY=AGENTCLI_API_KEY_PRIMARY
# clifwrap install agentcli
```

```toml
[[providers.agent.accounts]]
name = "primary"
env = { AGENTCLI_API_KEY = "env:AGENTCLI_API_KEY_PRIMARY" }

[[providers.agent.accounts]]
name = "backup"
env = { AGENTCLI_API_KEY = "env:AGENTCLI_API_KEY_BACKUP" }
```

Behavior:

- Interactive TUI (`agent`, `agent "prompt"`) uses `tty-exec` — picks an account, then execs the real binary with a live terminal (no output buffering).
- Headless print mode (`agent -p "..."`, `--print`, optional `--force` / `--output-format`) gets buffered failover on rate-limit / auth errors.
- Management commands (`login`, `logout`, `status`, `about`, `models`, `mcp`, …) pass through unchanged.

If the tool's status command reports auth identity but not remaining quota, proactive pick stays off unless you add your own `status_command` / `usage` metadata.

See [docs/configuration.md](docs/configuration.md) for the full reference.

### Capacity control
Expand Down Expand Up @@ -118,6 +150,8 @@ Command costs are estimates, not billing truth. SearchCLI and ScrapeCLI defaults

When capacity control picks an account, retries for that request stay on that account. A retryable upstream error will not burn through your reserve accounts.

With `proactive_pick` enabled (default when usage or `status_command` is configured), clifwrap also picks the best starting account before the first upstream call so a depleted default does not burn a retry. See [docs/configuration.md](docs/configuration.md#proactive-starting-account).

### Account management from the wrapped CLI

Upstream auth still works as usual:
Expand Down Expand Up @@ -313,6 +347,7 @@ scrapecli search "example"
## Docs

- [Configuration](docs/configuration.md)
- [Provider authoring](docs/provider-authoring.md)
- [CLI reference](docs/cli-reference.md)
- [Built-in provider catalog](docs/provider-catalog.md)
- [Migration from cli-fallback-wrapper](docs/migration.md)
Expand Down
159 changes: 159 additions & 0 deletions docs/brainstorms/2026-07-25-agent-native-requirements.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,159 @@
---
date: 2026-07-25
topic: agent-native
---

## Summary

Give clifwrap a shared agent-native contract so coding agents and CI can drive it without guessing flags or parsing ad-hoc JSON. Unify existing `--json` output under one envelope, document stable exit codes, and add `--dry-run` on mutating clifwrap commands.

---

## Problem Frame

clifwrap already exposes `--json` on several operational commands (`doctor`, `status`, `config validate`, `account list`, queue commands), but each command returns a different top-level shape. Exit codes are meaningful in practice (`--check` fails on issues) yet undocumented as a taxonomy agents can rely on. Mutating commands (`install`, `uninstall`, `account add`, queue replay) have no machine-readable output and no dry-run path, so agents and CI must either scrape human text or execute side effects to learn what would happen.

The extensibility track improves provider validation and authoring; it deliberately deferred a full agent-native surface. This track closes that gap for the commands operators and automation already use daily.

---

## Key Decisions

- **Contract plus dry-run, not full spec compliance.** Ship a unified JSON envelope, documented exit codes, and dry-run on mutating clifwrap commands. Defer machine-readable schema discovery and ai-native-cli-spec coverage across every command.
- **Same contract for agents and CI.** One envelope and exit-code table serves interactive agent sessions and pipeline scripts; no separate "CI mode."
- **Wrap, don't redesign.** Existing per-command payloads move under a `data` field inside the envelope rather than redesigning each command's internal structure.
- **Additive migration for envelope.** When `--json` is used, nest the current payload under `data` and add envelope fields alongside. Human (non-JSON) output stays unchanged. Breaking top-level JSON keys ship only with a documented migration note in the release changelog.
- **Dry-run applies to clifwrap-owned mutations only.** Install, uninstall, account mutations, and queue mutations support dry-run. Wrapped upstream CLIs are out of scope.

---

## Requirements

**JSON envelope**

- R1. Every command that supports `--json` emits a top-level envelope with at least: success indicator, command identity, and a `data` field holding the command's payload.
- R2. On failure, the envelope includes a structured error object with a stable machine-readable code and a human-readable message. Secret values never appear in JSON output.
- R3. Envelope fields are consistent across commands so a single parser can handle doctor, status, validate, account list, and queue output.

**Exit codes**

- R4. clifwrap documents a small stable exit-code taxonomy (success, usage/argument error, config error, operational check failure, internal error) mapped to real command behavior.
- R5. `--check` on doctor and status continues to exit nonzero on check failures; the taxonomy explains which code applies.
- R6. JSON error responses and process exit codes align: an agent reading only stdout can determine outcome; an agent reading only exit code can branch without parsing JSON.

**Dry-run**

- R7. `install`, `uninstall`, `account add`, `account remove`, `queue run`, and `queue drop` accept `--dry-run` and perform no persistent side effects when it is set.
- R8. Dry-run combined with `--json` returns the envelope with a `data` field describing planned changes (targets, accounts, queue items) without applying them.
- R9. Dry-run exits 0 when the operation would succeed; exits with the appropriate taxonomy code when it would fail (missing backup, invalid config, queue item not found).

**Documentation and discoverability**

- R10. `docs/operations.md` and generated CLI reference document the envelope shape, exit-code table, and which commands support `--json` and `--dry-run`.
- R11. `clifwrap doctor --json` includes envelope version or contract version in metadata so agents can detect contract generation.

---

## Key Flows

- F1. CI preflight before deploy
- **Trigger:** Pipeline runs health checks before releasing or rotating accounts.
- **Actors:** CI script, clifwrap CLI
- **Steps:** Run `config validate --json` → parse envelope → run `doctor --json --check` → branch on exit code and `data.issues`.
- **Outcome:** Pipeline fails fast with a parseable reason, no custom grep rules per subcommand.
- **Covered by:** R1, R4, R5, R6

- F2. Agent plans shim install without side effects
- **Trigger:** Coding agent needs to verify install targets before modifying PATH shims.
- **Actors:** Agent, clifwrap CLI
- **Steps:** Run `install <app> --dry-run --json` → read planned shim and backup paths from `data` → confirm with user → run without dry-run.
- **Outcome:** Agent never replaces a binary based on guessed flags.
- **Covered by:** R7, R8, R9

- F3. Agent replays queue safely
- **Trigger:** Capacity recovered; automation needs to drain queued work.
- **Actors:** Agent or cron job
- **Steps:** `queue list --json` → `queue run --dry-run --json` to preview → `queue run --json` to execute.
- **Outcome:** No duplicate replay; agent sees item count and outcomes in envelope.
- **Covered by:** R1, R7, R8

---

## Acceptance Examples

- AE1. Envelope on success
- **Covers:** R1, R3
- **Given:** Valid config and `clifwrap config paths --json`.
- **When:** Command completes successfully.
- **Then:** stdout is JSON with success indicator true, command identity set, and prior path fields nested under `data`.

- AE2. Envelope on validation failure
- **Covers:** R2, R6
- **Given:** Config references a missing env ref.
- **When:** `clifwrap config validate --json` runs.
- **Then:** Envelope reports failure, `data` includes findings, exit code matches the config-error category in the documented taxonomy.

- AE3. Dry-run install
- **Covers:** R7, R8, R9
- **Given:** `searchcli` is on PATH and not yet shimmed.
- **When:** `clifwrap install searchcli --dry-run --json` runs.
- **Then:** Exit 0; `data` describes shim target and backup path; no shim file is written.

- AE4. Check mode exit code
- **Covers:** R5, R6
- **Given:** Doctor reports validation warnings and `--check` is set.
- **When:** `clifwrap doctor --json --check` runs.
- **Then:** Exit code is the documented operational-check-failure code; envelope success indicator is false or `data` reflects check failure per contract rules.

---

## Success Criteria

- A CI script can drive `validate` and `doctor --check` using only envelope parsing and documented exit codes, without per-command JSON shape knowledge.
- An agent can plan `install` and `queue run` via `--dry-run --json` without causing side effects.
- Operations docs list every `--json` and `--dry-run` command in one place.

---

## Scope Boundaries

**Deferred for later**

- Machine-readable command/schema discovery command (`reference`, `schema`, or equivalent)
- Full ai-native-cli-spec compliance on every subcommand
- `--json` on wrapped upstream CLI passthrough invocations
- Reliability track (proactive quota pick, circuit breakers, failover event log)
- Security track (keychain refs, credential tiering)
- Context-routing track (project profiles, per-account config dirs)

**Outside this product's identity**

- HTTP API or MCP server wrapping clifwrap
- Agent-specific natural-language CLI (clifwrap remains a standard POSIX-style CLI)

---

## Dependencies / Assumptions

- Extensibility track (`config validate`, `doctor` validation section) lands first or in parallel; agent-native work extends those JSON outputs rather than replacing validation behavior.
- Existing `--json` consumers are few (early v0.1); additive nesting under `data` is acceptable with a changelog note.
- Human-readable command output remains the default; JSON is opt-in per command via `--json`.
- Generated `docs/cli-reference.md` continues to be the published command inventory.

---

## Outstanding Questions

**Deferred to planning**

- Exact envelope field names and whether `ok`/`success` follows agent-native-design or an internal convention.
- Whether `account import` dry-run reuses the existing apply-without-apply pattern or gains an explicit `--dry-run` flag aligned with other mutators.
- Contract version bump policy: semver in envelope meta vs clifwrap package version.

---

## Sources / Research

- Prior brainstorm: `docs/brainstorms/2026-07-25-extensibility-requirements.md` (deferred agent-native track)
- Repo: `src/clifwrap/__main__.py` (existing `--json` flags), `docs/operations.md`, `docs/cli-reference.md`
- External: [agent-native-design](https://agents365-ai.github.io/agent-native-design/), [CLI Spec](https://clispec.dev/), codex-quota and gh CLI `--json` patterns from improvement research
142 changes: 142 additions & 0 deletions docs/brainstorms/2026-07-25-extensibility-requirements.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,142 @@
---
date: 2026-07-25
topic: extensibility
---

## Summary

Make adding and customizing CLI providers in clifwrap safe, predictable, and self-documenting. Ship tiered provider validation, fix catalog override semantics for list fields, and publish a provider authoring guide with a minimal template catalog entry.

---

## Problem Frame

clifwrap already supports declarative providers through `src/clifwrap/providers.toml` and user `config.toml` overrides, but extensibility breaks down in practice. Operators discover misconfiguration only when a wrapped command fails at runtime. List fields such as retry patterns use all-or-nothing merge semantics, so a partial override silently replaces the entire catalog list. Only `searchcli` and `scrapecli` exist as built-in examples, and there is no dedicated guide for authoring a new provider.

The user prioritized extensibility over reliability, security, agent automation, and public-adoption polish for this track. Research against tools like GitHub CLI, teamclaude, AWS credential chains, and the CLI Spec surfaced validation and merge semantics as the highest-leverage gaps without introducing a plugin framework yet.

---

## Key Decisions

- **Validation before runtime, not a plugin system.** Extend existing `config validate` and `doctor` surfaces rather than building pip-based provider discovery or Python hooks in v1 of this track.
- **Warn by default; fail on explicit check.** Offline and air-gapped setups should get warnings for unreachable usage probes. `--check` (and CI-style usage) exits nonzero on validation failures, matching existing `doctor --check` behavior.
- **Template provider, not a third production provider.** Add a minimal catalog entry whose purpose is documentation-by-example, not shipping another real upstream CLI.
- **Append-aware list merge for catalog overrides.** User config must be able to extend catalog list fields without replacing them wholesale unless the user explicitly opts into replace semantics.

---

## Requirements

**Provider validation**

- R1. `clifwrap config validate` reports provider-level issues beyond TOML parsing: missing accounts, unresolvable env refs, invalid merge declarations, and malformed provider metadata.
- R2. `clifwrap doctor` includes a provider validation section that summarizes per-provider health and links each issue to a remediation command.
- R3. A provider probe mode validates live behavior without mutating config: usage URL or `status_command` returns expected JSON fields (`remaining`, `limit`, optionally `used`), and retry rule entries are syntactically valid.
- R4. Validation messages name the provider, the config location, and the next command to fix the issue.
- R5. Without `--check`, unreachable usage endpoints produce warnings. With `--check`, they produce errors and a nonzero exit code.

**Config merge semantics**

- R6. List fields inherited from the catalog (retry patterns, passthrough commands, and similar) support append-by-default when the user supplies entries in `config.toml`.
- R7. Users can opt into full replacement for a list field when they intend to drop catalog defaults entirely.
- R8. Merge rules for list fields are documented in `docs/configuration.md` with before/after examples.

**Authoring and discovery**

- R9. A provider authoring guide explains how to add a catalog provider vs a config-only generic provider, required fields, and common failure modes.
- R10. A minimal template catalog provider demonstrates every optional field with safe defaults and example placeholder URLs.
- R11. Generated `docs/provider-catalog.md` includes the template provider so operators can discover it from published docs.

---

## Key Flows

- F1. Author validates a new provider before first use
- **Trigger:** Operator adds accounts and overrides for a provider in `config.toml`.
- **Actors:** Operator, clifwrap CLI
- **Steps:** Run `config validate` → fix static issues → run `doctor --check` → run provider probe → install shim and test wrapped command.
- **Outcome:** First real upstream invocation succeeds or fails for upstream reasons, not local misconfiguration.
- **Covered by:** R1, R2, R3, R4, R5

- F2. Operator extends catalog retry rules without losing defaults
- **Trigger:** Operator adds one custom retry pattern for a built-in provider.
- **Actors:** Operator
- **Steps:** Add pattern in user config → validate → confirm merged provider includes catalog defaults plus the new pattern.
- **Outcome:** Failover behavior extends rather than resets.
- **Covered by:** R6, R7, R8

---

## Acceptance Examples

- AE1. Missing env ref
- **Covers:** R1, R4
- **Given:** An account references `env:MISSING_VAR` that is unset and has no `env_command`.
- **When:** The operator runs `clifwrap config validate`.
- **Then:** Output names the provider and account and suggests setting the variable or adding `env_command`.

- AE2. Offline validation
- **Covers:** R5
- **Given:** Usage URL is unreachable and the operator runs `doctor` without `--check`.
- **When:** Static validation passes but live probe fails.
- **Then:** Exit code is 0 with a warning; with `--check`, exit code is nonzero.

- AE3. Partial retry pattern override
- **Covers:** R6, R7
- **Given:** Catalog defines retry patterns A and B; user adds pattern C without a replace directive.
- **When:** Config is loaded and merged.
- **Then:** Effective patterns are A, B, and C. With explicit replace, effective patterns are only what the user specified.

---

## Success Criteria

- A new provider can be configured and validated locally before the first wrapped invocation.
- Operators can extend catalog list defaults without reading merge implementation source.
- `docs/provider-authoring.md` is sufficient for a contributor to propose a catalog entry in a PR without ad-hoc guidance.

---

## Scope Boundaries

**Deferred for later**

- Python plugin or pip-discovered provider packages
- External provider fragments in `~/.config/clifwrap/providers.d/` (may follow if template + validation prove insufficient)
- Proactive quota gating, 429-vs-quota failover distinction, and switch audit JSONL (reliability track)
- keyring-first credential storage (security track)
- Full agent-native CLI surface (`schema` command, `--json` on every data command) (automation track)
- Windows shim install strategy (platform track)

**Outside this product's identity**

- Local HTTP proxy architecture (CliRelay-style) — clifwrap remains a process shim, not a proxy server
- Web management dashboard for accounts

---

## Dependencies / Assumptions

- Existing catalog in `src/clifwrap/providers.toml` and merge logic in `src/clifwrap/config.py` remain the extension mechanism.
- `doctor` and `config validate` already exist and are the right UX anchors; this track extends them rather than introducing parallel commands.
- Template provider uses anonymized placeholder hosts consistent with `scripts/check_anonymity.py` constraints.
- Operators may run validation offline; live probes must not be mandatory for basic config edits.

---

## Outstanding Questions

**Deferred to planning**

- Exact syntax for list replace vs append (dedicated table key, operator prefix, or nested object).
- Whether provider probe runs one account or all enabled accounts by default.
- Whether template catalog provider ships enabled in the built-in catalog or is documented as a copy-paste starting point only.

---

## Sources / Research

- Repo: `src/clifwrap/config.py` (`merged_provider` list merge behavior), `src/clifwrap/__main__.py` (`config validate`, `doctor`), `docs/configuration.md`
- External: [CLI Spec](https://clispec.dev/) (schema, dry-run, scriptable exits), GitHub CLI `auth status` pattern, teamclaude quota-vs-rate-limit distinction, AWS credential precedence chain
- Prior exploration: improvement research covered reliability, security, and agent-native CLI as separate deferred tracks
Loading
Loading