feat(provider): add Devin CLI provider - #179
Closed
ademiwa wants to merge 1 commit into
Closed
Conversation
Add a `devin` provider backed by the local Devin CLI in non-interactive print mode. Provider selection via `--provider devin` / `CLAWPATCH_PROVIDER=devin`, with Devin-backed `map`, `review`, `fix`, `revalidate`, and `doctor --provider devin`. - Read-only enforcement via `--permission-mode auto` (review/revalidate) and `--permission-mode accept-edits` (fix), using the Devin runtime's own permission system rather than prompt-only directives - Prompt delivery via `--print --prompt-file <path>` to a temp file - JSON output parsed with the shared extractor (same approach as grok) - Reasoning effort injected into the prompt text (the Devin CLI has no `--reasoning-effort` flag as of v3000.2.17) - Model pass-through with `--model`; when unset, Devin uses its own configured default - Timeout: 300s default, override with `CLAWPATCH_DEVIN_TIMEOUT_MS` - Ambient env inherited (Devin uses config-file auth, like grok/cursor/ opencode; strict env isolation would break credential discovery) - Devin provider docs in `docs/providers.md`, `README.md`, `docs/spec.md` - Helper tests for prompt construction, timeout, and failure messages Also fixes cross-platform path expectations in the Claude env test block by using `path.join` instead of forward-slash string literals.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ffe908324c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| }); | ||
| }); | ||
|
|
||
| describe("Devin provider helpers", () => { |
There was a problem hiding this comment.
These tests cover prompt text, timeouts, and failure messages but never invoke the provider or assert its constructed CLI arguments. Consequently, regressions in the safety-critical auto versus accept-edits selection, prompt-file delivery, or model forwarding can ship while this suite remains green; add focused read-only and fix argument tests, preferably around an extracted argument builder.
AGENTS.md reference: AGENTS.md:L34-L40
Useful? React with 👍 / 👎.
clawsweeper
Bot
added
rating: 🦪 silver shellfish
Contributor
|
Codex review: needs real behavior proof before merge. Reviewed August 2, 2026, 2:22 PM ET / 18:22 UTC. ClawSweeper reviewWhat this changesAdds a local Devin CLI provider for Clawpatch map, review, fix, revalidate, and doctor workflows, with registry, helper-test, and provider-documentation updates. Merge readiness⛔ Blocked until real behavior proof is added - 7 items remain This remains a useful but unproven new provider feature: current main does not contain Devin support, while the PR still lacks coverage for its safety-critical command arguments and inspectable after-fix runtime proof. It also needs maintainer confirmation that this provider-specific credential and permission contract belongs in core. Priority: P3 Review scores
Verification
How this fits togetherClawpatch selects a named local coding-agent provider, writes the workflow prompt to a temporary file, invokes the provider CLI, and schema-validates its JSON response. This PR adds Devin at that boundary, where the selected permission mode determines whether a workflow is intended to be read-only or able to edit the checkout. flowchart LR
A[Clawpatch command] --> B[Provider selection]
B --> C[Devin provider]
C --> D[Temporary prompt file]
D --> E[Devin CLI permission mode]
E --> F[JSON response]
F --> G[Schema validation]
G --> H[Workflow result]
Decision needed
Why: Tests and runtime proof can validate the implementation, but they cannot determine whether this durable provider, credential, and safety commitment belongs in core. Before merge
Findings
Agent review detailsSecurityNeeds attention: The patch adds a third-party CLI execution and permission boundary, but the selected safety arguments are neither directly tested nor backed by inspectable live behavior proof. Review metrics
Merge-risk optionsMaintainer options:
Technical reviewBest possible solution: Keep the provider addition narrow, add focused tests for the exact Devin argument lists, attach redacted live proof for doctor plus a successful read-only workflow, and merge only after maintainers approve the resulting provider and credential contract. Do we have a high-confidence way to reproduce the issue? Not applicable: this PR adds a new provider rather than fixing established broken behavior. Its claimed live integration remains unproven because no inspectable successful Devin-backed Clawpatch run is attached. Is this the best way to solve the issue? Unclear: the local-provider shape matches existing integrations, but core acceptance depends on maintainer approval of Devin’s ambient-auth and permission-mode contract; the present patch also needs direct command-argument coverage. Full review comments:
Overall correctness: patch is incorrect AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning high; reviewed against 4eed9cd13c15. LabelsLabel justifications:
EvidenceSecurity concerns:
What I checked:
Likely related people:
Rank-up movesOptional improvements that raise the rating; they are not merge blockers.
Rating scale
Overall follows the weaker of proof and patch quality. Workflow
HistoryReview history (4 earlier review cycles)
|
clawsweeper
Bot
added
rating: 🧂 unranked krab
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a
devinprovider backed by the local Devin CLI in non-interactive print mode. Provider selection via--provider devin/CLAWPATCH_PROVIDER=devin, with Devin-backedmap,review,fix,revalidate, anddoctor --provider devin.Follows the same conventions as the
claude(#93) andgrokproviders: asrc/providers/devin.tsmodule exportingdevinProvider+devinTesting, registered insrc/provider.ts, with helper tests insrc/provider.test.tsand documentation indocs/providers.md,README.md, anddocs/spec.md.Motivation
Devin CLI (Cognition's agent CLI) is a coding-harness CLI that fits clawpatch's provider model (shells out to a local agent CLI, parses structured JSON output). Adding it expands the set of supported review/fix backends alongside
codex,claude,grok,opencode,pi, andcursor.Implementation details
--permission-mode: review and revalidate use--permission-mode auto(the Devin runtime auto-approves read-only tools only and never grants writes in print mode). Fix uses--permission-mode accept-edits. This is real tool-level enforcement, not a prompt-only directive — the same approach opencode uses withOPENCODE_PERMISSIONand grok uses with--disallowed-tools.devin --print --prompt-file <path> --permission-mode <mode>.extractJsonhelper (same approach asgrok,opencode,pi— the Devin CLI has no native--json-schemaflag).--reasoning-effortflag (verified against v3000.2.17), so the requested effort level is injected into the prompt text.--model <model>is passed through when configured; when unset, Devin uses its own configured default (~/.config/devin/config.jsonon macOS/Linux,%APPDATA%\devin\config.jsonon Windows).grok,cursor,opencode, andpi. Strict env isolation (asclaudedoes) would break Devin's credential discovery. This is consistent with the dominant provider pattern (4 of 5 comparable providers use ambient env).CLAWPATCH_DEVIN_TIMEOUT_MSorCLAWPATCH_PROVIDER_TIMEOUT_MS.checkmethod validates availability viadevin --version.Also includes a minor cross-platform fix: the Claude env test block in
src/provider.test.tsnow usespath.joininstead of forward-slash string literals for path expectations, so those tests pass on Windows as well as Unix.Testing performed
pnpm typecheck— passed (0 errors)pnpm lint— passed (0 warnings, 0 errors, 113 files, 125 rules)pnpm format:check— changed files pass (pre-existing CRLF format issues on 181 other files are unrelated to this PR)pnpm test— 871 passed, 22 skipped, 0 failed (full suite on the combined working tree)pnpm build— passed (clean compile)providerByName("devin")registry test: passeddevin --version→devin 3000.2.17,--print,--prompt-file,--model,--permission-modeall confirmed)Test plan
pnpm typecheck && pnpm lint && pnpm format:check && pnpm test && pnpm build)clawpatch doctor --provider devindetects the Devin CLI (requiresdevin auth loginfirst)clawpatch review --provider devin --limit 1produces structured findingsclawpatch map --source agent --provider devinproduces feature records