Skip to content

feat: add fail-closed cross-client capability negotiation - #7

Merged
DivyamTalwar merged 1 commit into
mainfrom
codex/orrery-capability-negotiation
Sep 5, 2026
Merged

DivyamTalwar merged 1 commit into
mainfrom
codex/orrery-capability-negotiation

Conversation

@DivyamTalwar

Copy link
Copy Markdown
Owner

Summary

Adds a read-only, versioned capability planner that evaluates a requested portability contract before configuration or adapter writes:

  • ranks all six supported clients by advisor-isolation strength and per-agent effort binding
  • emits machine-readable loss codes for every refused client
  • supports deterministic client subsets and project/user adapter destinations
  • preserves the requested-vs-observed boundary: static compatibility never claims live enforcement

This PR is independently based on e8a9604c8ac2b15de76d48080604c9151e76ec1d.

Competitive rationale and provenance

Tools such as AgentSync, MCPM, and Vercel Skills make cross-client installation convenient. Large agent catalogs such as wshobson/agents use generated adapters and graceful degradation. Orrery's differentiated gap is deciding before installation whether a target can represent a requested security/effort contract without silent loss.

The implementation is original and derived only from Orrery's existing CLIENT_PROFILES registry. No competitor code or data was copied. Host semantics remain linked to primary documentation such as Claude Code subagents and VS Code custom agents.

Verification

  • 4 focused planner tests: weakest contract, fail-closed OS+effort requirement, mechanism threshold, deterministic subset/user paths
  • planner is included in bun run test
  • docs include CLI usage, JSON contract, tie-breaking, and evidence limits
  • authoritative Linux/macOS CI is required before review

Evidence limit

The planner reads the versioned static registry. It does not probe installed host versions and explicitly refuses to present a compatible declaration as proof of the model, effort, or sandbox used at runtime.

Copy link
Copy Markdown
Owner Author

Verification evidence

Latest head: 07c2c1bdd68acd289571c86dc8cc6278c937696c

  • CI run 32663093159, job 97252106887: success — frozen install + full bun run ci
  • Compatibility run 32663093128: success
    • Ubuntu job 97252106835: typecheck + complete test suite
    • macOS job 97252106710: typecheck + complete test suite
  • Focused evidence: 4 new planner tests cover fail-closed losses, isolation thresholds, deterministic ranking, subsets, and scope paths.

The result is static capability evidence only; it deliberately does not claim live host enforcement.

Copy link
Copy Markdown
Owner Author

Competitive-review validation

Latest head 07c2c1bdd68acd289571c86dc8cc6278c937696c is green: CI 32663093159; Compatibility 32663093128 on Ubuntu and macOS.

The design adapts the fail-closed capability-matrix pattern observed across AgentSync, wshobson/agents, and anywhere-agents, while preserving ORRERY's stricter exact-model/effort/isolation semantics. No third-party code was copied. Static capability evidence remains explicitly distinct from live-host proof.

Per the requested workflow, this PR remains open and must not be merged.

@DivyamTalwar
DivyamTalwar force-pushed the codex/orrery-capability-negotiation branch from 76f60a9 to d1e86c9 Compare September 5, 2026 08:20
@DivyamTalwar

Copy link
Copy Markdown
Owner Author

Merge-order note — conflicts with #8.

Both this PR and #8 rewrite the same "test" line in package.json to append their own test file. Each is clean against main on its own, so GitHub reports both as mergeable — the conflict only appears after the first one lands.

The dangerous resolution is the quiet one: keep one test file, drop the other, and CI stays green because the missing tests simply never run.

Whichever merges second needs its test script to list both files:

tools/capability-planner.test.ts tools/tool-surface-review.test.ts

Ping me and I'll rebase the second one.

@DivyamTalwar
DivyamTalwar force-pushed the codex/orrery-capability-negotiation branch from d1e86c9 to 27d1fca Compare September 5, 2026 08:26
@DivyamTalwar

Copy link
Copy Markdown
Owner Author

Conflict resolved — rebased onto main (now carrying #8, #9, #15).

The conflict was the one flagged above: both this PR and #8 rewrote the same "test" line. Resolved by keeping both scripts and both test files rather than letting either drop out:

"plan": "bun tools/capability-planner.ts",
"tools:review": "bun tools/tool-surface-review.ts --baseline tools/schema/tools.policy.json",
"test": "bun test ... tools/capability-planner.test.ts tools/tool-surface-review.test.ts"

Verified the resolution actually holds rather than just compiling — bun test reports 82 tests across 6 files, so capability-planner.test.ts and tool-surface-review.test.ts are both executing. A resolution that silently dropped one would still have been green, which was the risk.

bun run ci green: VERIFY PASSED three-role · 82 pass / 0 fail · validate · sbom (18 files, 0 runtime deps) · release-check. Authorship is divyamtalwar15@gmail.com for author and committer.

@DivyamTalwar
DivyamTalwar merged commit 2c94f36 into main Sep 5, 2026
4 checks passed
@DivyamTalwar
DivyamTalwar deleted the codex/orrery-capability-negotiation branch September 5, 2026 08:27
DivyamTalwar added a commit that referenced this pull request Sep 5, 2026
Problem
Orrery gained three read-only diagnostics in three separate pull requests — the
capability planner (#7), the tool-surface review (#8), and the semantic
compatibility doctor (#9). Each arrived with a sentence appended wherever its own
change happened to touch the README, so two of them ended up as orphan paragraphs
below the development gate and the third existed only as a comment inside a code
block.

Read individually they look like three unrelated utilities. They are not: they are
the same capability at three different moments — before you choose a client, after
you install, and whenever the tool surface moves. That framing was nowhere on the
page, so the most checkable thing about this project was also the least visible.

Approach
Add "Interrogate it yourself" between "What it will not do" and "When not to use
this", so the honesty block reads in one run: what it refuses to do, how you can
check that for yourself, and when you should not use it at all.

The section is organised by moment rather than by tool, and each entry leads with
what the command *refuses* to do, because that is the part with value. A path-only
portability matrix would report six clients as working; the planner names the ones
that would quietly downgrade advisor isolation and refuses them. The doctor
separates what it observed on disk from what it cannot know about runtime
behaviour. The review turns "the digest moved" into whether the thing that moved
can take a new argument, hold state, or has dropped a `readOnlyHint`.

It closes by inverting the usual precedence: if a diagnostic disagrees with the
README, the README is wrong.

Verification
Every documented invocation was run, not copied from a pull request description.
`plan` with the documented flags returns `bestFit: codex` and refuses cursor and
the remaining clients. `tools:review` returns `changed: false`,
`permissionExpansion: false`. `doctor` reports schema 2 with `readOnlyMechanism`
and per-role adapter state. `bun run ci` is green.

Impact
Documentation only.
DivyamTalwar added a commit that referenced this pull request Sep 5, 2026
Problem
Four pull requests merged into main without a changelog entry between them: the
semantic compatibility doctor (#9), the explainable tool-surface review (#8), the
capability planner (#7), and the toolchain bump (#15). The Unreleased section
described only the Astra lane, so the three most user-visible additions since 0.6.0
existed in the code and in the README but nowhere in the release record.

A changelog that silently omits four merges is worse than no changelog, because it
is read as complete.

Approach
Document all four, then cut Unreleased as 0.7.0 and bring the three declared
versions into agreement. Minor rather than patch: the release adds a second native
implementation lane and three read-only diagnostics. Nothing is removed and no
interface breaks, so it is not a major.

Each entry records what the thing refuses to do, since that is the part with value:
the planner refuses hosts that would silently weaken requested effort or advisor
isolation; the review turns "the digest moved" into whether the change is
permission-bearing; the doctor no longer reports a marker-only or weakened adapter
as healthy. The toolchain entry records why the bump was applied by hand, so the
next person does not reopen the Dependabot question.

Verification
`bun run ci` is green: verify.sh three-role checks pass, 82 tests across 6 files,
validate, sbom (now emitting orrery-0.7.0.cdx.json, 18 packaged files, 0 runtime
dependencies), and release-check. `bun run tag:check -- v0.7.0` passes, so the tag
and all three manifests agree.

Impact
The release record now matches what shipped. Ready to tag v0.7.0 once merged.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant