diff --git a/.codex/INSTALL.md b/.codex/INSTALL.md index d1971beec..c463d523a 100644 --- a/.codex/INSTALL.md +++ b/.codex/INSTALL.md @@ -1,35 +1,37 @@ # Installing AgentOps for Codex -AgentOps 3.3 uses one canonical checkout and source symlinks. Do not use the -old curl installer or a Codex plugin cache. - -## Installation +Use native Codex and the shell with zero mandatory AgentOps skills: ```bash -brew tap boshu2/agentops https://github.com/boshu2/homebrew-agentops -brew install agentops -git clone https://github.com/boshu2/agentops.git ~/.local/share/agentops -cd ~/.local/share/agentops -ao skills link +go install github.com/boshu2/agentops/cli/cmd/ao@latest +ao quick-start +ao demo ``` -`ao skills link` creates source links under `~/.agents/skills` and every -detected runtime skill root, including `~/.codex/skills`. +Give Codex a concrete outcome and the repository's acceptance checks. A fresh +context reviews the completed change. No RPI invocation, skill bundle, hook or +`ao init` is required. -## Verification +## Optional guidance -Restart Codex and confirm AgentOps skills are visible (for example `/plan` or -`/quickstart`). - -## Update +From a source checkout, link only the skills you want to make discoverable: ```bash +git clone https://github.com/boshu2/agentops.git ~/.local/share/agentops cd ~/.local/share/agentops -git pull --ff-only -ao skills link +ao skills link --skill test --skill refactor --dry-run +ao skills link --skill test --skill refactor ``` -## Migration from 3.x +The command links canonical sources into `~/.agents/skills` and detected +runtime roots, including `~/.codex/skills`. Restart Codex if it snapshots its +inventory at startup. It does not remove existing skills or replace foreign +entries. No-selector `ao skills link` still links the full catalog. + +Managed plugins and npx full bundles remain supported optional installations. +Choose one source for a skill to avoid duplicate discovery. Updating a source +checkout updates its existing links; repeat the same selectors to restore them. -See [docs/MIGRATION.md](../docs/MIGRATION.md). Remove any old -`~/.codex/plugins/cache/agentops-marketplace` install before linking. +See [installation and updates](../docs/install-day2-ops.md) for Homebrew, +building development features from source, full bundles and removal, and +[migration](../docs/MIGRATION.md) for legacy installations. diff --git a/AGENTS.md b/AGENTS.md index ab6deb796..7acdb13b2 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -2,19 +2,24 @@ Detailed workflow mechanics: [docs/agent-workflow-reference.md](docs/agent-workflow-reference.md). -AgentOps is the operations layer for agentic engineering: skills and evidence +AgentOps is the operations layer for agentic engineering: tools, optional skills and evidence contracts that make one coding-agent change independently judgeable. Your tracker, Git, and coding agents keep owning work, history, and execution; AgentOps joins them as a federated integration graph and adds the judgment -step, one RPI traversal at a time: +step. The default is native execution with zero mandatory AgentOps skills: ```text -RPI charter -> on-demand Plan -> Implement and checks -> fresh Validate -> finish +Accepted intent -> native implementation and checks -> fresh independent judgment -> finish ``` No fresh independent judgment over the exact subject means the experiment is not proven. +Use the existing issue or conversation, edit the smallest useful change, run +the repository's checks, and have a fresh context judge acceptance. Load a +skill only for a concrete uncertainty or an explicitly selected workflow. +Native goals supply continuity; BD keeps work and handoffs. Neither needs RPI. + ## Repository map and mechanics | Path | What it is | @@ -147,7 +152,7 @@ Skill logic ships in Go via `ao`; `skills/*/scripts/**/*.py`. Skill tests retain their documented exemption (ADR-0016, gate-enforced). -## Lean RPI operating charter +## Native execution and optional Lean RPI operating charter Own the authorized outcome through finish. Use the existing accepted intent and scope; a clear trivial change needs no Plan, Recall or Learn worksheet. Take the @@ -178,19 +183,15 @@ out-of-scope change or failed acceptance means `FAIL`. Repair known findings within authority and real bounds, then revalidate the changed exact subject. Persist machine evidence only for a caller request or declared consumer. -[Memory](skills/memory/SKILL.md) is optional and on demand: recall applicable -reviewed external topic pages, or separately budget mining/learning and curation. -BD owns work/status/handoffs, Git content, and native/CASS systems episodes. -Reuse existing topic pages; entries give applicability, action, support, limits -and invalidation. One incident supports a narrow observation; stronger rules -need stronger evidence. Preserve rare useful constraints and legacy `.agents/` -evidence; no blind TTL/deletion. Learning may remove rules and no-change is valid. -Benefit requires later work evidence, not saved pages. This lean path accepts -public or already-cleared trial inputs only and claims no native enforcement -for restricted sources. Protected external drafts and exact independent support +[Memory](skills/memory/SKILL.md) optionally recalls reviewed external topic pages +or separately mines and curates them. Its contract owns support, applicability, +invalidation and preservation; no blind TTL/deletion. Benefit needs later work +evidence. Public or already-cleared inputs only: this path supplies no native +restricted-source enforcement. Protected external drafts and independent support and destination-disclosure review precede Git import (ADR-0016). -The [RPI skill](skills/rpi/SKILL.md) owns the charter; the +The [RPI skill](skills/rpi/SKILL.md) packages this charter when explicitly selected; +it is not a prerequisite for native execution or independent review. The [architecture reference](docs/architecture/rpi-traversal.md) owns exact evidence semantics. Optional outer-goal guidance and the grandfathered fixed-dispatch reference adapter stay outside the native core. No scheduler or new AO command diff --git a/GOALS.md b/GOALS.md index 47de1eee0..47991176f 100644 --- a/GOALS.md +++ b/GOALS.md @@ -21,7 +21,7 @@ explicit behavior 1. **Behavior before activity.** The caller-owned intent states the active behavior, acceptance examples where useful, non-goals, evidence, and bounded write scope before implementation begins. -2. **Bounded implementation.** Implement makes and checks the change, repairing +2. **Bounded implementation.** The native agent makes and checks the change, repairing known defects directly within the accepted scope and allowance. Aggregate budgets and delivery authority remain with the caller or selected runtime. 3. **Fresh judgment.** PASS requires explicit, distinct author and validator @@ -32,10 +32,10 @@ explicit behavior 5. **Honest uncertainty.** Mutation, incomplete changed-path coverage, missing identity, or missing proof returns NOT_PROVEN. Proven scope or acceptance failure returns FAIL. -6. **Sovereign proof.** When requested, Validate atomically writes +6. **Sovereign proof.** When requested, the fresh reviewer uses AO to atomically write content-addressed JSON to caller-selected protected external non-Git storage. Preserve legacy evidence under owner policy. Provenance is optional audit. -7. **Stop boundary.** RPI owns the authorized outcome, using Plan only when +7. **Stop boundary.** The native agent owns the authorized outcome, planning only when needed and revising approach within unchanged acceptance. Known defects get direct repair; a causal stall gets at most one bounded fresh helper inside the existing allowance. Finish when accepted and freshly validated, or stop @@ -46,7 +46,8 @@ explicit behavior ## Structural constraints -- Core hard dependencies are only `rpi -> {plan, implement, validate}`; these +- Native execution requires zero AgentOps skills. When selected, RPI's + core hard dependencies are only `rpi -> {plan, implement, validate}`; these capability references do not require invoking every skill on every change. - Memory, Learn and all strategy/factory/specialist skills are optional. - Core schemas contain no retry, budget, queue, claim, lease, admission, @@ -54,7 +55,7 @@ explicit behavior - The pure manifest and verdict helpers make no Git, tracker, queue, network, release, or delivery call. - Deterministic `ao gate check` reports repository check success or failure only. -- Semantic validation is a skill responsibility, not a CLI state machine. +- Semantic validation belongs to a fresh reviewer; Validate is optional guidance. ## Gates @@ -88,8 +89,9 @@ evidence for them. The selected CDLC (Context Delivery Lifecycle) contract maintains external context/environment around disposable agents; it neither trains weights nor -promises deterministic inference. Discovery (with Plan as shaping owner), -Implement and Validate operate within caller-owned intent and allowance. +promises deterministic inference. Discovery, implementation and independent +validation operate within caller-owned intent and allowance. Their corresponding +skills are optional. Optional Memory recall and mining are skill guidance, not a new runtime or automatic private-source authorization; evolve restoration belongs only to T25. No unavailable entrypoint is enabled by this document. diff --git a/PRACTICE-REGISTRY.md b/PRACTICE-REGISTRY.md index a5227ad61..8655a3167 100644 --- a/PRACTICE-REGISTRY.md +++ b/PRACTICE-REGISTRY.md @@ -149,33 +149,29 @@ an AI agent with a finite context window, not just a human team. The constraint changes the *size* and *shape* of the artifacts those practices produce. The practices themselves do not change. -## What the agent-context constraint changes - -1. **Granularity** — smaller modules, shorter ADRs, tighter bounded contexts, - more snapshots. Each artifact must fit one context window. -2. **Linkage over inclusion** — artifacts forward-reference rather than - embed; agents follow links lazily. -3. **Discovery surfaces are mandatory** — an `INDEX.md` is not optional, it - is the entry point. A skill without an index is invisible. -4. **Drift is fatal** — human teams could tolerate documentation lag; agents - follow stale signposts blindly, so a stale signpost actively misleads. - The forcing functions DevOps invented for code and infrastructure now - apply to *every* signpost. -5. **Verification is the spec** — TDD/BDD wasn't optional before; it's - load-bearing now because agents trust the verifier, not the prose. - A slice without a snapshot or property test is wishful thinking. -6. **Knowledge compounds, or it actively rots** — agents are stateless - between sessions. Without a structured way to capture and recall what was - tried, why it changed course, what evidence mattered, every session - starts from zero. The practices that worked for human teams still work, - but the cost of skipping them is now immediate, not eventual. -7. **Intent must be linked through phases** — humans can carry intent in - memory, side conversations, and organizational context; agents cannot rely - on any of that. Discovery intent must become planning packet, council - verdict, implementation contract, validation evidence, and handoff without - depending on chat history. In AgentOps, the packet is the linked-intent - object: it carries the objective, constraints, evidence rules, and - provenance anchors from one phase to the next. +## Applying the registry to native execution + +Select practices for the concrete engineering problem; this catalog is not a +mandatory workflow. For this repository's native default: + +- Use BDD examples to clarify ambiguous behavior, including consequential edge + cases. A small clear change can use the existing issue and tests directly. +- Use DDD to keep work tracking, source content, execution, evidence and memory + with their existing owners. Introduce an abstraction only for a real boundary. +- Prefer small batches, direct repairs and tests at the changed behavior's seam. + A behavioral fix benefits from a right-reason failing test; a documentation + edit does not need a manufactured RED or an executable test of every sentence. +- Link to current sources and retrieve them on demand. An index earns its cost + when it solves an observed discovery problem; no per-task INDEX or packet is + required. +- Run the repository's required integration checks and obtain one fresh review + of the exact accepted change. Review depth follows risk and unresolved facts. +- Reuse reviewed external knowledge when applicable. Preserve support and + limitations; later task evidence must establish benefit. Capturing a lesson + does not itself prove compounding. + +A native goal and existing tracker can preserve intent and handoff facts. +Discovery, planning, implementation and review do not each need a new artifact. ## What does NOT change @@ -192,33 +188,16 @@ practices produce. The practices themselves do not change. ## Implication for this repo -Every artifact under `skills/`, `hooks/`, `evals/`, `docs/`, `schemas/`, -`scripts/`, and `cli/` is exactly one of: - -1. A **bounded context** — a Primitive in our domain vocabulary - (`skills/domain/references/primitive.md`) -2. A **spec** — a Slice, ADR-shaped and BDD-flavored - (`skills/domain/references/slice.md`) -3. A **verification** — a test, a snapshot, an eval suite, a hook gate -4. A **discovery surface** — an INDEX, a manifest, or a signpost README - -Each artifact is sized to fit one agent context window. Each forward-links -to others. Each has a forcing function (snapshot hash, property test, schema -check, eval suite) that fails when reality drifts from intent. - -Those four artifact kinds are not isolated. Each belongs to an intent chain: -source material -> packet -> briefing -> verdict or plan -> execution packet --> validation evidence -> handoff or learning. If an artifact changes the -work, it must make clear what prior artifact it consumed and what next phase, -artifact, or operator it feeds. Provenance and trace are how the chain remains -auditable after the session that created it is gone. - -When you build something new in this repo, ask: which of those four kinds is -this? If the answer is "none", the artifact is probably misplaced. If the -answer is "all four", the artifact is too large. - -That is the practice. AgentOps is its codification under the -AI-context-limited constraint. +Keep artifacts with their source owner: code, tests, contracts, and only the +navigation or evidence a real consumer needs. Before adding process material, +name that consumer, the decision it supports, the observed defect and the +condition under which it can be retired. A new program written only to consume +an otherwise unnecessary artifact does not justify it. + +Use the smallest acceptance-advancing action. None of these practices requires +installing or invoking an AgentOps skill; skills are optional guidance for +specific uncertainties. The registry does not prove that every practice helps +every task or that the skill library outperforms native model judgment. ## Practice slugs (canonical registry) diff --git a/PRODUCT.md b/PRODUCT.md index 4886d6dce..21e68fb4d 100644 --- a/PRODUCT.md +++ b/PRODUCT.md @@ -1,5 +1,5 @@ --- -last_reviewed: 2026-09-09 +last_reviewed: 2026-09-10 --- # AgentOps @@ -16,7 +16,7 @@ handoffs — and the product boundary is deliberately small. The standard traversal through the graph is: ```text -RPI charter -> on-demand Plan -> Implement and checks -> fresh Validate -> finish +Accepted intent -> native implementation and checks -> fresh independent judgment -> finish ``` ## Proven floor @@ -49,7 +49,13 @@ AgentOps for delivery permission. ## Product surfaces -RPI carries the operating charter; four skills are loaded as needed: +The default uses the coding agent and shell with zero mandatory AgentOps skills. +The repository brief supplies acceptance, source boundaries and required checks. +`ao` supplies deterministic tools on demand. A native goal can maintain +continuity; the caller's tracker keeps work and handoffs. + +The existing skill library remains optional. Select guidance when a task needs +it; an explicit RPI workflow packages these operations: | Skill | Responsibility | |---|---| @@ -67,8 +73,9 @@ and swarms may provide roles and dispatch. None is a correctness or lifecycle authority. The `ao` CLI supplies deterministic repository checks and generic read-only or -record helpers where useful. Semantic validation belongs to the Validate skill, -not a CLI state machine. +record helpers where useful. A fresh native reviewer owns semantic validation; +the optional Validate skill packages that method. A CLI cannot issue semantic +acceptance from successful checks alone. ## Sovereign evidence @@ -81,8 +88,9 @@ and ledger availability are never required for validity. ## Selected Context Delivery Lifecycle contract -The lean [RPI charter](skills/rpi/SKILL.md) owns the authorized outcome through -finish with on-demand Plan, Implement, Validate and optional Memory. Known +Native execution owns the authorized outcome through finish. The optional +[RPI charter](skills/rpi/SKILL.md) supplies on-demand Plan, Implement, Validate +and Memory when selected. Known failures get direct repair; evidence can revise an approach within unchanged acceptance and scope. A genuine causal stall admits at most one bounded helper, never a helper chain. Cheap checks precede fresh author-distinct final judgment; diff --git a/PROGRAM.md b/PROGRAM.md index 974407858..f69fd5acd 100644 --- a/PROGRAM.md +++ b/PROGRAM.md @@ -1,19 +1,19 @@ # Program -This repository self-hosts the lean RPI charter it ships (ADR-0017). PROGRAM is +This repository defaults to native execution with optional skills (ADR-0017). PROGRAM is repository execution guidance, not a product retry or delivery controller. ## Experiment unit One invocation consumes and produces: -1. one resolved caller-owned intent, using Plan only when needed and a content +1. one resolved caller-owned intent, planning only when needed and a content snapshot only when necessary; 2. bounded implementation and checks, with direct repair of known defects and evidence-based approach revision within unchanged acceptance; 3. one runtime-derived subject manifest and factual check receipts with complete or honestly incomplete changed-path proof; -4. fresh author-distinct Validate judgment over exact content, with fresh +4. fresh author-distinct native judgment over exact content, with fresh re-validation after acceptance-relevant repairs; 5. a subject-led report, with a durable verdict only when requested or required by a declared consumer. @@ -39,7 +39,7 @@ ordinary implementation authority. Use the cheapest targeted checks during implementation. Before reporting the complete change, run the ordinary deterministic suite appropriate to the final -surface and obtain one fresh Validate verdict. Record exactly what was checked +surface and obtain one fresh independent judgment. Record exactly what was checked and not checked. The repository may then commit, push, merge, release, or roll back through its diff --git a/README.md b/README.md index 33fa99ac9..f7192a931 100644 --- a/README.md +++ b/README.md @@ -1,22 +1,24 @@ # AgentOps AgentOps is the operations layer for agentic engineering. It is a set of -portable skills and evidence contracts that make one coding-agent change +deterministic tools, optional skills and evidence contracts that make one coding-agent change independently judgeable: the context that wrote the code does not get to declare it done. Your tracker keeps the work, Git keeps the history, and your coding agents keep running the execution; AgentOps joins them as a federated integration graph and adds the judgment step. A fresh context reads the exact change and returns `PASS`, `FAIL`, or `NOT_PROVEN`. The standard -path is one RPI traversal: +path uses your native coding agent with zero mandatory AgentOps skills: ```text -RPI charter -> on-demand Plan -> Implement and checks -> fresh Validate -> finish +Accepted intent -> native implementation and checks -> fresh independent judgment -> finish ``` -The lean [RPI charter](skills/rpi/SKILL.md) owns an authorized outcome through -finish: Plan on demand, direct repair of understood failures, cheap checks and -fresh final judgment. Evidence can revise an approach within unchanged outcome -and scope. A clear small edit needs no planning or memory worksheet. +Give the agent a concrete outcome, acceptance examples and the repository's +checks. Let it implement, repair understood failures and obtain a fresh review. +A native goal can carry continuity; your tracker keeps work and handoffs. +Finish at accepted work. A clear small edit needs no planning or memory worksheet. +The [RPI charter](skills/rpi/SKILL.md) remains an optional workflow, selected +when its guidance helps the task. [Memory](skills/memory/SKILL.md) offers on-demand recall and separately budgeted mining/curation over reviewed caller-selected external Markdown topic pages. @@ -29,14 +31,31 @@ legacy evidence preservation follow [ADR-0016](docs/adr/ADR-0016-state-tiers.md) ## Quickstart ```bash -npx skills@latest add boshu2/agentops --all -g +go install github.com/boshu2/agentops/cli/cmd/ao@latest +ao quick-start ``` -One command installs the skill bundle into every coding agent you use. The -skills run **inside your coding agent** (Claude Code, Codex, Cursor, …): type -`/rpi` in that agent's chat, or ask for `plan`, `implement`, `validate`, and -`learn` by name. Most skills need nothing beyond the coding agent; these need -more: +The native path needs no skill bundle, hook, `ao init`, or session bootstrap. +`ao quick-start` gives read-only guidance; `ao demo` prints a sample coding +task. Use `ao` for a specific check or evidence operation when useful. +See [installation](docs/install-day2-ops.md) for Homebrew and source builds. + +## Optional skill library + +From an AgentOps checkout, expose only guidance you want: + +```bash +ao skills link --skill test --skill refactor --dry-run +ao skills link --skill test --skill refactor +``` + +Skills run **inside your coding agent** (Claude Code, Codex, Cursor, …). +Linking makes them discoverable; it does not require invoking them. Select a +skill for a concrete task need. An explicit full install remains supported: +`npx skills@latest add boshu2/agentops --all -g`, or `ao skills link` without +selectors from a checkout. Existing installations are preserved. + +Most skills need nothing beyond the coding agent; these need more: | Skill | Needs | Why | |---|---|---| @@ -54,7 +73,7 @@ more: | `security` | `python3`, conditional | the composable suite and offline redteam surfaces run `security_suite.py` when that scan type is selected | | `cass` | `python3`, optional | `scripts/prompt_miner.py` mines repeated prompts; one of several selectable Scripts-table entries | -The plugin and `npx skills@latest add boshu2/agentops --all -g` install all 54 skills today, regardless of whether you have `python3` or `ao`. +The plugin and `npx skills@latest add boshu2/agentops --all -g` install the generated skill catalog, regardless of whether you have `python3` or `ao`. Ran it? Tell us what it judged. Open an issue, and paste the `verdict.v2` if you asked `validate` to persist one: @@ -62,7 +81,7 @@ you asked `validate` to persist one: ## Plugins (Claude Code / Codex) -Prefer a managed bundle that updates with the release: +For an explicit full managed bundle that updates with the release: ```bash # Claude Code @@ -74,14 +93,14 @@ codex plugin marketplace add boshu2/agentops codex plugin add agentops@agentops-marketplace ``` -Three install paths: +Three optional skill installation paths: - **npx / [skills.sh](https://skills.sh)**: universal; copies skills you can edit. - **Plugins**: a read-only bundle that stays current with the repo. - **Checkout + `ao skills link`**: source-tracked symlinks for contributors (see [Install and day-2 operations](docs/install-day2-ops.md)). -## Admission-control hooks (on by default) +## Optional admission-control hooks AgentOps ships a PreToolUse **policy dispatcher**: deterministic guards that block a small set of known-destructive commands (staging the private bead @@ -91,7 +110,10 @@ call; every block is one line. - **Claude Code plugin installs:** active automatically; nothing to run. - **npx / skills.sh copies:** run `~/.claude/skills/cc-hooks/scripts/install-hooks.sh` once. -- **git clone / brew:** run `scripts/install-policy-dispatch.sh` once. +- **git clone / brew:** opt in with `scripts/install-policy-dispatch.sh`. + +AO-only installation does not install hooks. The Claude plugin enables its +bundled hooks when that optional installation is selected. Disable anytime (`/plugin disable agentops`, or remove the two PreToolUse matchers from settings). Policy list and design: @@ -103,16 +125,14 @@ directories. ## Intent lives in a bead [Beads](https://github.com/steveyegge/beads) is the preferred tracker -(optional; `brew install beads`). Plan -writes [BDD](https://cucumber.io/docs/bdd/) acceptance and DDD [ubiquitous -language](https://martinfowler.com/bliki/UbiquitousLanguage.html) into the bead; -Implement builds against it; Validate judges a hashed snapshot under -`.agents/ao/intents/sha256/`. No beads? Plan shapes the caller's issue or chat -text and the runtime snapshots those bytes the same way. These are standalone -product-proof defaults; selected CDLC knowledge/disclosure evidence requires -protected external routing before storage (ADR-0016). - -`validate` runs in a fresh context from the author's model family by default: +(optional; `brew install beads`). Use [BDD](https://cucumber.io/docs/bdd/) +acceptance examples and DDD [ubiquitous language](https://martinfowler.com/bliki/UbiquitousLanguage.html) +when they clarify behavior and ownership boundaries. The native agent implements +that intent; a fresh reviewer judges exact content against it. An issue or +conversation also works. Snapshot mutable intent only when needed; new persisted +proof requires protected external non-Git storage (ADR-0016). + +Review runs in a fresh context from the author's model family by default: Codex reviews Codex work, and Claude reviews Claude work. Request `--cross-model [model]` in Validate or RPI to add a different-family reviewer; an unavailable requested leg leaves the combined result unproven. Review time @@ -150,13 +170,13 @@ Two factory stacks are supported: provisioning, skill visibility, and the evidence boundary. AgentOps does not wrap either factory or translate factory completion into -semantic PASS. When proof is required, a fresh `validate` context judges the +semantic PASS. When proof is required, a fresh reviewer judges the exact candidate and evidence. -## Optional: `ao` CLI +## Deterministic `ao` CLI -Deterministic checks, inspection, and skill linking. `fitness` and -`using-gc` call it directly; the rest of the skills work without it. Install +Deterministic checks, inspection, and optional skill linking. Dependency +requirements vary by skill as listed above. Install steps (Homebrew or `go install`), and `ao skills link` for tracking skills from a local checkout: [Install and day-2 operations](docs/install-day2-ops.md#maintainer--contributor-the-ao-binary). @@ -166,8 +186,8 @@ tracking skills from a local checkout: ### 1. The agent said it was done Same session that wrote the code also declared victory. AgentOps separates -authorship from judgment: `implement` produces a candidate; `validate` must -run in a fresh context and may use a different model. It issues `PASS`, +authorship from judgment: the coding agent produces a candidate; a reviewer +runs in a fresh context and may use a different model. It issues `PASS`, `FAIL`, or `NOT_PROVEN`. ### 2. One perspective rubber-stamped another @@ -175,7 +195,7 @@ run in a fresh context and may use a different model. It issues `PASS`, A single context can share blind spots with the author. Opt into [`idea-genie`](skills/idea-genie/SKILL.md) or [`council`](skills/council/SKILL.md) for sealed or multi-judge review. They return a report; an author-distinct -[`validate`](skills/validate/SKILL.md) context issues the binding result. +reviewer issues the binding result, optionally guided by [`validate`](skills/validate/SKILL.md). ### 3. Acceptance drifted mid-flight @@ -185,14 +205,14 @@ without silently changing acceptance. Validation binds to that accepted intent. ### 4. Nobody can replay what was judged -Chat scrolls away. When replay or automation needs durable evidence, `validate` +Chat scrolls away. When replay or automation needs durable evidence, the reviewer writes a content-addressed `verdict.v2` in caller-selected protected external non-Git storage, with checked scope, omissions, and evidence refs. Existing `.agents/` proof remains preserved under owner policy. Plain JSON. No hosted service required. Interactive validation does not create one unless requested. -## Core skills +## Optional workflow skills | Skill | Job | |---|---| diff --git a/cli/README.md b/cli/README.md index 4634cc9d7..8d6c50113 100644 --- a/cli/README.md +++ b/cli/README.md @@ -1,11 +1,11 @@ # ao — AgentOps CLI `ao` supplies deterministic repository utilities and evidence inspection. It -is the optional checks/linking CLI of the AgentOps operations layer; the -semantic protocol lives in the skills: +is the checks/linking CLI of the AgentOps operations layer. Native execution +requires zero AgentOps skills; a fresh reviewer judges the result: ```text -RPI → Plan → Implement → fresh Validate → repair to convergence → report +Accepted intent → native implementation and checks → fresh independent judgment → finish ``` The CLI does not own retries, queues, work claims, Git delivery, release, @@ -39,7 +39,7 @@ make test ``` Add deterministic utilities only when they do not become lifecycle or delivery -authorities. Keep semantic judgment in the Validate skill, optional verdict +authorities. Keep semantic judgment with a fresh reviewer (Validate guidance is optional), verdict persistence with declared consumers, and external delivery in the consumer repository. diff --git a/cli/cmd/ao/native_onboarding_test.go b/cli/cmd/ao/native_onboarding_test.go new file mode 100644 index 000000000..494651acb --- /dev/null +++ b/cli/cmd/ao/native_onboarding_test.go @@ -0,0 +1,56 @@ +package main + +import ( + "context" + "os" + "os/exec" + "strings" + "testing" + "time" +) + +// Given an installed binary and an empty home/consumer checkout, the default +// front doors need neither skills nor setup and leave both directories empty. +func TestNativeOnboardingWithoutSkillsOrSetupWrites(t *testing.T) { + bin := aoBinary(t) + for _, args := range [][]string{{"--help"}, {"quick-start"}, {"quick-start", "--help"}, {"demo"}, {"demo", "--quick"}, {"demo", "--concepts"}, {"demo", "--help"}, {"init", "--help"}, {"robot-docs"}, {"robot-docs", "--help"}} { + t.Run(strings.Join(args, "_"), func(t *testing.T) { + home, consumer := t.TempDir(), t.TempDir() + ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second) + defer cancel() + cmd := exec.CommandContext(ctx, bin, args...) + cmd.Dir = consumer + cmd.Env = []string{"HOME=" + home, "PATH=" + home, "CODEX_HOME=" + home} + output, err := cmd.CombinedOutput() + if err != nil { + t.Fatalf("native command: %v\n%s", err, output) + } + text := string(output) + for _, want := range []string{"native", "optional"} { + if !strings.Contains(strings.ToLower(text), want) { + t.Errorf("native entry missing %q:\n%s", want, text) + } + } + for _, forbidden := range []string{"invoke the Validate skill", "Run the RPI skill", "run this first", "RPI -> Plan -> Implement", "Semantic judgment belongs to the Validate skill"} { + if strings.Contains(text, forbidden) { + t.Errorf("default entry requires workflow %q:\n%s", forbidden, text) + } + } + for _, dir := range []string{home, consumer} { + entries, err := os.ReadDir(dir) + if err != nil || len(entries) != 0 { + t.Fatalf("native entry changed %s: %v (err %v)", dir, entries, err) + } + } + }) + } +} + +func TestExplicitRPIDemoRemainsAvailable(t *testing.T) { + ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second) + defer cancel() + output, err := exec.CommandContext(ctx, aoBinary(t), "demo", "--rpi").CombinedOutput() + if err != nil || !strings.Contains(string(output), "AGENTOPS RPI DEMO") { + t.Fatalf("explicit full workflow unavailable: %v\n%s", err, output) + } +} diff --git a/cli/cmd/ao/root.go b/cli/cmd/ao/root.go index 144f05f07..627a02018 100644 --- a/cli/cmd/ao/root.go +++ b/cli/cmd/ao/root.go @@ -30,11 +30,14 @@ var rootCmd = &cobra.Command{ Version: version, Short: "AgentOps CLI: deterministic checks and evidence records", Long: `ao provides deterministic repository checks and generic evidence records. -Semantic judgment belongs to the Validate skill. Git, delivery, retries, work -ownership, and continuation belong to the caller and repository. +Your native coding agent owns the approach and completes accepted work. Fresh +independent judgment of the exact final change is required; workflow skills, +specialists and evidence persistence are optional. No skills, bootstrap or init +are required. Git, delivery, work ownership and continuation stay with the caller. For AI agents: - ao capabilities Machine-readable CLI contract (JSON) — run this first. + ao quick-start Native execution brief, without mandatory workflow skills. + ao capabilities Machine-readable CLI contract (JSON), when useful. ao robot-docs Paste-ready agent handbook. Append --json to any read-side command for structured output. diff --git a/cli/cmd/skill-trial-report/main.go b/cli/cmd/skill-trial-report/main.go index c21bedc3d..54e3e1cca 100644 --- a/cli/cmd/skill-trial-report/main.go +++ b/cli/cmd/skill-trial-report/main.go @@ -10,6 +10,7 @@ import ( "os" "strings" + "github.com/boshu2/agentops/cli/internal/evidence" "github.com/boshu2/agentops/cli/internal/skilltrial" ) @@ -22,16 +23,54 @@ func run(args []string, out, stderr io.Writer) error { flags := flag.NewFlagSet("skill-trial-report", flag.ContinueOnError) flags.SetOutput(stderr) var jobs, sessions values + var providers, verdicts, criteria values + var judgment evidence.JudgmentOptions + var profiles string var reward string flags.Var(&jobs, "job", "explicit [arm=]Harbor job directory (repeatable; arm is a caller label)") flags.Var(&sessions, "session", "additional explicit native Codex JSONL file (repeatable; never scans operator home)") flags.StringVar(&reward, "reward-key", "reward", "native verifier reward key; only 1 and 0 classify success and failure") + flags.StringVar(&judgment.Root, "root", "", "explicit subject directory for one independent judgment join") + flags.StringVar(&judgment.Manifest, "manifest", "", "expected subject-manifest.v1 file") + flags.StringVar(&judgment.BaseManifest, "base-manifest", "", "base manifest for deletions") + flags.StringVar(&judgment.Intent, "intent", "", "independent immutable acceptance file") + flags.StringVar(&judgment.EvidenceRoot, "evidence-root", "", "private non-Git root for original verdicts, receipts and reviewer transcripts") + flags.StringVar(&judgment.AuthorContextID, "author-context-id", "", "independent author identity matching an explicitly supplied native session") + flags.StringVar(&profiles, "required-profiles", "", "independent required-profiles JSON used by provenance verify-judgments") + flags.Var(&providers, "allowed-provider", "independently authorized provider (repeatable)") + flags.Var(&verdicts, "verdict", "original content-addressed verdict.v2 file (repeatable; missing required legs stay unproven)") + flags.Var(&criteria, "required-criterion", "caller-supplied acceptance criterion ID (repeat for the complete expected set; never derive from the verdict)") if err := flags.Parse(args); err != nil { return err } if flags.NArg() != 0 { return fmt.Errorf("unexpected positional arguments; use --job or --session") } + judgmentSelected := false + flags.Visit(func(f *flag.Flag) { + if f.Name != "job" && f.Name != "session" && f.Name != "reward-key" { + judgmentSelected = true + } + }) + if judgmentSelected { + for _, required := range []struct{ name, value string }{{"root", judgment.Root}, {"manifest", judgment.Manifest}, {"intent", judgment.Intent}, {"evidence-root", judgment.EvidenceRoot}, {"author-context-id", judgment.AuthorContextID}, {"required-profiles", profiles}} { + if required.value == "" { + return fmt.Errorf("judgment join requires --%s", required.name) + } + } + if len(providers) == 0 { + return fmt.Errorf("judgment join requires --allowed-provider") + } + if len(criteria) == 0 { + return fmt.Errorf("judgment join requires --required-criterion for the complete acceptance ID set") + } + var err error + judgment.Required, err = evidence.LoadJudgeProfiles(profiles) + if err != nil { + return err + } + judgment.AllowedProviders, judgment.Verdicts, judgment.RequiredCriteria = providers, verdicts, criteria + } inputs := make([]skilltrial.JobInput, 0, len(jobs)) for _, value := range jobs { arm, dir, ok := strings.Cut(value, "=") @@ -47,6 +86,10 @@ func run(args []string, out, stderr io.Writer) error { if err != nil { return err } + if judgmentSelected { + report.Work = skilltrial.InspectWork(report, judgment) + report.Limits = append(report.Limits, "Work joins one explicitly selected subject to its observed native author. Accepted requires completed execution and the existing verifier's satisfied independent PASS coverage over the complete caller-supplied acceptance criterion ID set; endpoint reward remains separate. A verified FAIL is retained as failed, while missing or invalid proof is not_proven. Association cannot choose among different session copies or multiple trials.", "Judgment verification binds original verdicts, subject/acceptance, receipts and native reviewer identities. The caller owns the complete required criterion ID set; the fresh reviewer maps those IDs to immutable intent and supporting evidence. Arbitrary criterion evidence references are not mechanically resolved. The reader issues no semantic verdict and establishes no complete billing or general skill benefit.") + } encoder := json.NewEncoder(out) encoder.SetIndent("", " ") return encoder.Encode(report) diff --git a/cli/cmd/skill-trial-report/work_test.go b/cli/cmd/skill-trial-report/work_test.go new file mode 100644 index 000000000..7e9e06664 --- /dev/null +++ b/cli/cmd/skill-trial-report/work_test.go @@ -0,0 +1,398 @@ +package main + +import ( + "bytes" + "encoding/json" + "os" + "path/filepath" + "strings" + "testing" + + "github.com/boshu2/agentops/cli/internal/evidence" + "github.com/boshu2/agentops/cli/internal/skilltrial" +) + +type workFixture struct { + args []string + root, proof, author, reviewer, receiptPath string + receipt evidence.JudgmentReceipt + verdict map[string]any +} + +func workPut(t *testing.T, path string, raw []byte) { + t.Helper() + if err := os.MkdirAll(filepath.Dir(path), 0700); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(path, raw, 0600); err != nil { + t.Fatal(err) + } +} + +func workJSON(t *testing.T, path string, value any) []byte { + t.Helper() + raw, err := evidence.Canonical(value) + if err != nil { + t.Fatal(err) + } + workPut(t, path, raw) + return raw +} + +func newWorkFixture(t *testing.T) *workFixture { + t.Helper() + f := &workFixture{root: t.TempDir(), proof: t.TempDir()} + workPut(t, filepath.Join(f.root, "answer.go"), []byte("package answer\nconst Value = 42\n")) + manifest, err := evidence.BuildManifest(f.root, []string{"answer.go"}, nil, nil, nil) + if err != nil { + t.Fatal(err) + } + manifestPath, intentPath, profilesPath := filepath.Join(f.proof, "manifest.json"), filepath.Join(f.proof, "intent"), filepath.Join(f.proof, "profiles.json") + workJSON(t, manifestPath, manifest) + intent := []byte("value: answer.Value is 42\nscope: changes are limited to answer.go\n") + workPut(t, intentPath, intent) + profile := evidence.JudgeProfile{ID: "review", Runtime: "codex", Model: "gpt-test", Family: "openai", Effort: "high"} + workJSON(t, profilesPath, map[string]any{"profiles": []evidence.JudgeProfile{profile}}) + f.author, f.reviewer = filepath.Join(f.proof, "author.jsonl"), filepath.Join(f.proof, "reviewer.jsonl") + workPut(t, f.author, []byte("{\"type\":\"session_meta\",\"payload\":{\"id\":\"author\"}}\n{\"type\":\"event_msg\",\"payload\":{\"type\":\"task_complete\"}}\n")) + native := []byte("{\"type\":\"session_meta\",\"payload\":{\"id\":\"reviewer\",\"model\":\"gpt-test\",\"model_provider\":\"openai\",\"reasoning_effort\":\"high\"}}\n{\"type\":\"event_msg\",\"payload\":{\"type\":\"task_complete\"}}\n") + workPut(t, f.reviewer, native) + zero, yes, no := 0, true, false + f.receipt = evidence.JudgmentReceipt{Version: "1", Requested: profile, SubjectManifestDigest: manifest.Digest, AcceptanceDigest: evidence.Hash(intent), AuthorContextID: "author", Transcript: evidence.TranscriptBinding{Path: f.reviewer, Start: 0, End: int64(len(native)), SHA256: evidence.Hash(native)}, ExitCode: &zero, TimedOut: &no, Truncated: &no, CleanupVerified: &yes, Omissions: []string{}} + f.verdict = map[string]any{"schema_version": "verdict.v2", "subject_manifest_digest": manifest.Digest, "acceptance_digest": evidence.Hash(intent), "author_context_id": "author", "validator_context_id": "reviewer", "freshness_attestation": map[string]any{"source": "runtime", "attester_identity": "native-test"}, "verdict": "PASS", "criteria": []any{ + map[string]any{"id": "value", "result": "PASS", "evidence_refs": []string{"check:value-42"}}, + map[string]any{"id": "scope", "result": "PASS", "evidence_refs": []string{"check:scope"}}}, "findings": []any{}, "checked": []string{"answer.go"}, "not_checked": []string{}, "validated_at": "2026-09-10T19:00:00Z"} + f.args = []string{"--session", f.author, "--root", f.root, "--manifest", manifestPath, "--intent", intentPath, "--evidence-root", f.proof, "--author-context-id", "author", "--required-profiles", profilesPath, "--allowed-provider", "openai", "--required-criterion", "value", "--required-criterion", "scope"} + return f +} + +func (f *workFixture) seal(t *testing.T) []string { + t.Helper() + f.receiptPath = filepath.Join(f.proof, "receipt.json") + raw := workJSON(t, f.receiptPath, f.receipt) + f.verdict["evidence_refs"] = []string{"judgment-receipt:" + f.receiptPath + "#sha256=" + evidence.Hash(raw)} + delete(f.verdict, "artifact_digest") + digest, err := evidence.Digest(f.verdict) + if err != nil { + t.Fatal(err) + } + f.verdict["artifact_digest"] = digest + path := filepath.Join(f.proof, digest+".json") + workJSON(t, path, f.verdict) + return append(append([]string{}, f.args...), "--verdict", path) +} + +func workRun(t *testing.T, args []string) map[string]any { + t.Helper() + var out, stderr bytes.Buffer + if err := run(args, &out, &stderr); err != nil { + t.Fatalf("report: %v; %s", err, stderr.String()) + } + var report map[string]any + if err := json.Unmarshal(out.Bytes(), &report); err != nil { + t.Fatal(err) + } + return report +} + +func TestNativeWorkRequiresIndependentProof(t *testing.T) { + f := newWorkFixture(t) + without := workRun(t, []string{"--session", f.author}) + if work, _ := without["work"].(map[string]any); work != nil && work["status"] == "accepted" { + t.Fatal("execution became acceptance") + } + with := workRun(t, f.seal(t)) + work, _ := with["work"].(map[string]any) + if work == nil || work["status"] != "accepted" || work["execution"] != "completed" || work["association"] != "standalone_session" { + t.Fatalf("work: %+v", work) + } + if with["sessions"].([]any)[0].(map[string]any)["copies"].([]any)[0].(map[string]any)["accounting"].(map[string]any)["usage"] != nil { + t.Fatal("missing usage fabricated") + } +} + +func TestNativeWorkPreservesIndependentFailure(t *testing.T) { + f := newWorkFixture(t) + f.verdict["verdict"] = "FAIL" + f.verdict["criteria"].([]any)[0].(map[string]any)["result"] = "FAIL" + work := workRun(t, f.seal(t))["work"].(map[string]any) + if work["status"] != "failed" || work["judgments"].(map[string]any)["satisfied"] != false { + t.Fatalf("coverage replaced FAIL: %+v", work) + } +} + +func TestNativeWorkInvalidOrMissingProofNeverAccepts(t *testing.T) { + cases := []struct { + name string + before func(*testing.T, *workFixture) + after func(*testing.T, *workFixture) + want string + }{ + {name: "wrong subject", before: func(_ *testing.T, f *workFixture) { f.receipt.SubjectManifestDigest = strings.Repeat("a", 64) }, want: "subject_mismatch"}, + {name: "wrong acceptance", before: func(_ *testing.T, f *workFixture) { f.receipt.AcceptanceDigest = strings.Repeat("a", 64) }, want: "acceptance_mismatch"}, + {name: "no freshness", before: func(_ *testing.T, f *workFixture) { f.verdict["freshness_attestation"] = nil }, want: "verdict_integrity"}, + {name: "self review", before: func(_ *testing.T, f *workFixture) { f.verdict["validator_context_id"] = "author" }, want: "distinct nonempty context"}, + {name: "missing criteria", before: func(_ *testing.T, f *workFixture) { f.verdict["criteria"] = []any{} }, want: "criteria must be nonempty"}, + {name: "missing criterion evidence", before: func(_ *testing.T, f *workFixture) { + f.verdict["criteria"].([]any)[0].(map[string]any)["evidence_refs"] = []string{} + }, want: "unproven criterion"}, + {name: "unchecked acceptance", before: func(_ *testing.T, f *workFixture) { f.verdict["not_checked"] = []string{"scope"} }, want: "verdict_integrity"}, + {name: "omitted evidence", before: func(_ *testing.T, f *workFixture) { f.receipt.Omissions = []string{"check log unavailable"} }, want: "omitted_evidence"}, + {name: "reviewer timeout", before: func(_ *testing.T, f *workFixture) { yes := true; f.receipt.TimedOut = &yes }, want: "incomplete_runtime"}, + {name: "missing reviewer", after: func(t *testing.T, f *workFixture) { + if err := os.Remove(f.reviewer); err != nil { + t.Fatal(err) + } + }, want: "no such file"}, + {name: "missing receipt", after: func(t *testing.T, f *workFixture) { + if err := os.Remove(f.receiptPath); err != nil { + t.Fatal(err) + } + }, want: "receipt_integrity"}, + {name: "tampered receipt", after: func(t *testing.T, f *workFixture) { workPut(t, f.receiptPath, []byte("{}")) }, want: "receipt_integrity"}, + {name: "tampered native span", after: func(t *testing.T, f *workFixture) { workPut(t, f.reviewer, []byte("{}\n")) }, want: "transcript"}, + {name: "changed content", after: func(t *testing.T, f *workFixture) { + workPut(t, filepath.Join(f.root, "answer.go"), []byte("package answer\nconst Value = 0\n")) + }, want: "subject content no longer matches"}, + {name: "unknown author", before: func(_ *testing.T, f *workFixture) { + for i := range f.args { + if f.args[i] == "--author-context-id" { + f.args[i+1] = "different" + } + } + }, want: "author_session_missing"}, + {name: "native NOT_PROVEN", before: func(_ *testing.T, f *workFixture) { f.verdict["verdict"] = "NOT_PROVEN" }, want: "verdict_not_pass"}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + f := newWorkFixture(t) + if tc.before != nil { + tc.before(t, f) + } + args := f.seal(t) + if tc.after != nil { + tc.after(t, f) + } + work := workRun(t, args)["work"].(map[string]any) + raw, err := json.Marshal(work) + if err != nil { + t.Fatal(err) + } + if work["status"] != "not_proven" || !strings.Contains(string(raw), tc.want) { + t.Fatalf("wanted %s: %s", tc.want, raw) + } + }) + } +} + +func TestNativeWorkRequiresCompleteUnambiguousExecution(t *testing.T) { + for _, tc := range []struct{ name, event, want string }{ + {"started", `{"type":"task_started"}`, "unfinished"}, + {"aborted", `{"type":"task_aborted"}`, "failed"}, + {"uninterpreted abort after completion", "{\"type\":\"task_complete\"}}\n{\"type\":\"event_msg\",\"payload\":{\"type\":\"turn_aborted\"}", "unknown"}, + {"error with completion marker", `{"type":"task_complete","error":{"message":"local execution failed"}}`, "failed"}, + } { + t.Run(tc.name, func(t *testing.T) { + f := newWorkFixture(t) + workPut(t, f.author, []byte("{\"type\":\"session_meta\",\"payload\":{\"id\":\"author\"}}\n{\"type\":\"event_msg\",\"payload\":"+tc.event+"}\n")) + work := workRun(t, f.seal(t))["work"].(map[string]any) + if work["status"] != "not_proven" || work["execution"] != tc.want || work["judgments"].(map[string]any)["satisfied"] != true { + t.Fatalf("execution and judgment conflated: %+v", work) + } + }) + } + t.Run("different cumulative copies", func(t *testing.T) { + f := newWorkFixture(t) + copyPath := filepath.Join(f.proof, "other-copy.jsonl") + raw, err := os.ReadFile(f.author) + if err != nil { + t.Fatal(err) + } + workPut(t, copyPath, append(raw, '\n')) + args := append(f.seal(t), "--session", copyPath) + work := workRun(t, args)["work"].(map[string]any) + if work["status"] != "not_proven" || work["association"] != "ambiguous" || len(work["author_evidence"].([]any)) != 2 { + t.Fatalf("copy selected: %+v", work) + } + }) +} + +func TestNativeWorkJoinsOneTrialAndKeepsAllAttempts(t *testing.T) { + f := newWorkFixture(t) + job := t.TempDir() + workPut(t, filepath.Join(job, "result.json"), []byte(`{"n_total_trials":3}`)) + terminal := `{"started_at":"2026-09-10T14:00:00Z","finished_at":"2026-09-10T14:00:05Z","verifier_result":{"rewards":{"reward":1}}}` + workPut(t, filepath.Join(job, "passing", "result.json"), []byte(terminal)) + workPut(t, filepath.Join(job, "failed", "result.json"), []byte(strings.ReplaceAll(terminal, `"reward":1`, `"reward":0`))) + workPut(t, filepath.Join(job, "unfinished", "config.json"), []byte(`{}`)) + raw, err := os.ReadFile(f.author) + if err != nil { + t.Fatal(err) + } + workPut(t, filepath.Join(job, "passing", "agent", "sessions", "native.jsonl"), raw) + args := append(f.seal(t), "--job", "native="+job) + first := workRun(t, args) + work := first["work"].(map[string]any) + if work["status"] != "accepted" || work["association"] != "trial" || filepath.Base(work["trial_directory"].(string)) != "passing" { + t.Fatalf("trial association: %+v", work) + } + counts := first["jobs"].([]any)[0].(map[string]any)["counts"].(map[string]any) + if counts["success"] != float64(1) || counts["failed"] != float64(1) || counts["unfinished"] != float64(1) { + t.Fatalf("attempts lost: %+v", counts) + } + a, err := json.Marshal(first) + if err != nil { + t.Fatal(err) + } + b, err := json.Marshal(workRun(t, args)) + if err != nil { + t.Fatal(err) + } + if !bytes.Equal(a, b) { + t.Fatal("replay changed") + } + workPut(t, filepath.Join(job, "failed", "agent", "sessions", "native.jsonl"), raw) + work = workRun(t, args)["work"].(map[string]any) + if work["status"] != "not_proven" || work["association"] != "ambiguous" { + t.Fatalf("retry chosen: %+v", work) + } +} + +func TestNativeWorkMissingRequiredLegAndPartialFlags(t *testing.T) { + f := newWorkFixture(t) + work := workRun(t, f.args)["work"].(map[string]any) + if work["status"] != "not_proven" { + t.Fatalf("missing required verdict accepted: %+v", work) + } + var out, stderr bytes.Buffer + if err := run([]string{"--session", f.author, "--root", f.root}, &out, &stderr); err == nil || !strings.Contains(err.Error(), "requires --manifest") { + t.Fatalf("partial judgment flags: %v", err) + } +} + +func TestNativeWorkRequiresCompleteCallerCriterionSet(t *testing.T) { + for _, tc := range []struct { + name string + mutate func(*workFixture) + problem string + }{ + {"partial omission", func(f *workFixture) { f.verdict["criteria"] = f.verdict["criteria"].([]any)[:1] }, "missing_required_criterion: scope"}, + {"duplicate verdict ID", func(f *workFixture) { f.verdict["criteria"].([]any)[1].(map[string]any)["id"] = "value" }, "duplicate_criterion: value"}, + {"unknown verdict ID", func(f *workFixture) { f.verdict["criteria"].([]any)[1].(map[string]any)["id"] = "other" }, "unexpected_criterion: other"}, + {"unknown expected ID", func(f *workFixture) { f.args[len(f.args)-1] = "other" }, "missing_required_criterion: other"}, + {"duplicate expected ID", func(f *workFixture) { f.args[len(f.args)-1] = "value" }, "criterion IDs must be nonempty, unpadded and unique"}, + {"empty expected ID", func(f *workFixture) { f.args[len(f.args)-1] = "" }, "criterion IDs must be nonempty, unpadded and unique"}, + } { + t.Run(tc.name, func(t *testing.T) { + f := newWorkFixture(t) + tc.mutate(f) + work := workRun(t, f.seal(t))["work"].(map[string]any) + raw, err := json.Marshal(work) + if err != nil { + t.Fatal(err) + } + if work["status"] != "not_proven" || !strings.Contains(string(raw), tc.problem) { + t.Fatalf("incomplete or ambiguous criteria accepted: %s", raw) + } + if strings.Contains(tc.problem, "_criterion:") { + legs := work["judgments"].(map[string]any)["legs"].([]any) + if len(legs) != 1 || legs[0].(map[string]any)["verdict"] != "PASS" { + t.Fatalf("original invalid leg lost: %s", raw) + } + } + }) + } +} + +func TestNativeWorkCannotAcceptWithoutCriterionExpectation(t *testing.T) { + f := newWorkFixture(t) + f.args = f.args[:len(f.args)-4] // The complete required-criterion input is absent. + args := f.seal(t) + var out, stderr bytes.Buffer + if err := run(args, &out, &stderr); err == nil || !strings.Contains(err.Error(), "requires --required-criterion") { + t.Fatalf("missing CLI criterion policy: %v", err) + } + profiles, err := evidence.LoadJudgeProfiles(filepath.Join(f.proof, "profiles.json")) + if err != nil { + t.Fatal(err) + } + options := evidence.JudgmentOptions{Root: f.root, Manifest: filepath.Join(f.proof, "manifest.json"), Intent: filepath.Join(f.proof, "intent"), EvidenceRoot: f.proof, AuthorContextID: "author", Required: profiles, AllowedProviders: []string{"openai"}, Verdicts: []string{args[len(args)-1]}} + report, err := skilltrial.Build(nil, []string{f.author}, "reward") + if err != nil { + t.Fatal(err) + } + work := skilltrial.InspectWork(report, options) + if work.Status != "not_proven" || work.Judgments == nil || !work.Judgments.Satisfied || !strings.Contains(strings.Join(work.Problems, ";"), "required_criteria_missing") { + t.Fatalf("legacy coverage without expected criteria became accepted: %+v", work) + } +} + +func TestNativeWorkKeepsCounterUncertaintySeparateFromExecution(t *testing.T) { + for _, tc := range []struct{ event, execution, status string }{ + {"task_complete", "completed", "accepted"}, + {"task_started", "unfinished", "not_proven"}, + } { + t.Run(tc.event, func(t *testing.T) { + f := newWorkFixture(t) + native := "{\"type\":\"session_meta\",\"payload\":{\"id\":\"author\"}}\n" + + "{\"type\":\"event_msg\",\"payload\":{\"type\":\"token_count\",\"info\":{\"total_token_usage\":{\"input_tokens\":100,\"output_tokens\":10}}}}\n" + + "{\"type\":\"event_msg\",\"payload\":{\"type\":\"token_count\",\"info\":{\"total_token_usage\":{\"input_tokens\":50,\"output_tokens\":5}}}}\n" + + "{\"type\":\"event_msg\",\"payload\":{\"type\":\"" + tc.event + "\"}}\n" + workPut(t, f.author, []byte(native)) + report := workRun(t, f.seal(t)) + work := report["work"].(map[string]any) + if work["status"] != tc.status || work["execution"] != tc.execution || strings.Contains(strings.Join(anyStrings(work["problems"].([]any)), ";"), "author_accounting_unverified") { + t.Fatalf("counter uncertainty blocked native execution evidence: %+v", work) + } + accounting := report["sessions"].([]any)[0].(map[string]any)["copies"].([]any)[0].(map[string]any)["accounting"].(map[string]any) + if len(accounting["diagnostics"].([]any)) != 1 || accounting["usage"].(map[string]any)["input_tokens"] != float64(50) { + t.Fatalf("unknown aggregate accounting hidden or added: %+v", accounting) + } + }) + } +} + +func anyStrings(values []any) []string { + result := make([]string, len(values)) + for i, value := range values { + result[i] = value.(string) + } + return result +} + +func TestNativeWorkMalformedAuthorStillFailsClosed(t *testing.T) { + f := newWorkFixture(t) + raw, err := os.ReadFile(f.author) + if err != nil { + t.Fatal(err) + } + workPut(t, f.author, append([]byte("{invalid\n"), raw...)) + work := workRun(t, f.seal(t))["work"].(map[string]any) + if work["status"] != "not_proven" || !strings.Contains(strings.Join(anyStrings(work["problems"].([]any)), ";"), "author_accounting_unverified") { + t.Fatalf("parse uncertainty was treated as usage-only: %+v", work) + } +} + +func TestNativeWorkPassingCodeCannotCompleteInterruptedTrial(t *testing.T) { + for _, tc := range []struct{ name, result, outcome, execution string }{ + {"timeout with passing code", `{"started_at":"2026-09-10T14:00:00Z","finished_at":"2026-09-10T14:00:05Z","exception_info":{"exception_type":"AgentTimeoutError"},"verifier_result":{"rewards":{"reward":1}}}`, "execution_error", "failed"}, + {"unfinished native trial", `{"started_at":"2026-09-10T14:00:00Z","finished_at":null,"verifier_result":{"rewards":{"reward":1}}}`, "unfinished", "unfinished"}, + } { + t.Run(tc.name, func(t *testing.T) { + f := newWorkFixture(t) + job := t.TempDir() + workPut(t, filepath.Join(job, "trial", "result.json"), []byte(tc.result)) + raw, err := os.ReadFile(f.author) + if err != nil { + t.Fatal(err) + } + workPut(t, filepath.Join(job, "trial", "agent", "sessions", "native.jsonl"), raw) + report := workRun(t, append(f.seal(t), "--job", job)) + work := report["work"].(map[string]any) + trial := report["jobs"].([]any)[0].(map[string]any)["trials"].([]any)[0].(map[string]any) + if work["status"] != "not_proven" || work["execution"] != tc.execution || work["judgments"].(map[string]any)["satisfied"] != true || trial["outcome"] != tc.outcome { + t.Fatalf("passing review/code rehabilitated execution: work=%+v trial=%+v", work, trial) + } + }) + } +} diff --git a/cli/docs/COMMANDS.md b/cli/docs/COMMANDS.md index 315bdb985..ed634db91 100644 --- a/cli/docs/COMMANDS.md +++ b/cli/docs/COMMANDS.md @@ -19,7 +19,7 @@ ### `ao demo` -Show the AgentOps product boundary: +Show a native coding-agent change from accepted behavior through checks, ``` ao demo [flags] @@ -30,14 +30,15 @@ ao demo [flags] ``` --concepts explain the product boundary -h, --help help for demo - --quick show the compact RPI example + --quick show the compact native example (the default) + --rpi show the optional full RPI workflow ``` --- ### `ao init` -Create the local intent and verdict evidence directories, then add one +Optional local evidence setup; native execution needs no init, skills or ``` ao init [flags] @@ -47,7 +48,7 @@ ao init [flags] ### `ao quick-start` -AgentOps is a small semantic evidence layer around agent work. +Use your native coding agent and shell to complete the accepted work. ``` ao quick-start [flags] @@ -1071,7 +1072,7 @@ ao skills graph [flags] #### `ao skills link` -Scan skills/ and create a live-tier symlink for every skill dir that has +Optionally install skills from a source checkout. Native execution needs no ``` ao skills link [flags] @@ -1080,9 +1081,10 @@ ao skills link [flags] **Flags:** ``` - --dest string Link into this single dir instead of the auto-detected roots (default: ~/.agents plus every installed runtime) - -h, --help help for link - --json Emit machine-readable JSON + --dest string Link into this single dir instead of the auto-detected roots (default: ~/.agents plus every installed runtime) + -h, --help help for link + --json Emit machine-readable JSON + --skill stringArray Install only this skill; repeat for more names (default: all skills) ``` #### `ao skills list` diff --git a/cli/internal/commands/demo/module.go b/cli/internal/commands/demo/module.go index 998bca031..c40b6caf7 100644 --- a/cli/internal/commands/demo/module.go +++ b/cli/internal/commands/demo/module.go @@ -41,47 +41,75 @@ func (Module) Contract() clicontract.CommandContract { } } -// Command builds the `ao demo` command with constructor-scoped --quick and -// --concepts flags. +// Command builds the native demo and its explicitly selected RPI alternative. func (Module) Command() *cobra.Command { var ( quick bool concepts bool + rpi bool ) command := &cobra.Command{ Use: "demo", - Short: "Show one RPI traversal", - Long: `Show the AgentOps product boundary: - - RPI -> Plan -> Implement -> fresh Validate -> repair to convergence -> report + Short: "Show native execution with optional workflow examples", + Long: `Show a native coding-agent change from accepted behavior through checks, +fresh independent judgment and delivery under repository policy. +No skills or bootstrap are required. Specialists and evidence persistence are +optional. Use --rpi to show the full workflow-skill alternative. The repository keeps its own Git, CI, tracker, release, and delivery policy.`, GroupID: "start", RunE: func(cmd *cobra.Command, _ []string) error { if concepts { return showConcepts(cmd.OutOrStdout()) } + if rpi { + return rpiDemo(cmd.OutOrStdout()) + } return quickDemo(cmd.OutOrStdout()) }, } - command.Flags().BoolVar(&quick, "quick", false, "show the compact RPI example") + command.Flags().BoolVar(&quick, "quick", false, "show the compact native example (the default)") command.Flags().BoolVar(&concepts, "concepts", false, "explain the product boundary") + command.Flags().BoolVar(&rpi, "rpi", false, "show the optional full RPI workflow") + command.MarkFlagsMutuallyExclusive("concepts", "rpi") return command } func showConcepts(w io.Writer) error { fmt.Fprintln(w, `AGENTOPS PRODUCT BOUNDARY -AgentOps shapes one behavior, runs one bounded implementation experiment, -obtains one fresh independent judgment over exact content, reports it, and -stops. Machine-readable verdict persistence is optional and consumer-driven. +The native coding agent owns the approach and completes accepted work using +repository instructions, ordinary tools and deterministic checks. A fresh +independent judgment over exact content is required for accepted completion; +using the Validate skill to obtain it is optional. No workflow skills or +bootstrap are required. Machine-readable verdict persistence is optional. -It does not own retries, budgets, queues, work ownership, Git, closure, release, -or delivery. Learn and multi-agent strategies are optional callers.`) +AgentOps does not own retries, budgets, queues, work ownership, Git, closure, release, +or delivery. Specialists and multi-agent strategies are optional callers.`) return nil } func quickDemo(w io.Writer) error { + fmt.Fprintln(w, `AGENTOPS NATIVE DEMO + +Request: a parser must reject an empty value while preserving valid inputs. +1. Read repository instructions and the parser's accepted behavior. +2. Add a discriminating regression, reproduce the failure, and repair the parser + with native coding-agent tools. Run the required package and repository checks. +3. Use ao gate check for deterministic facts; repair understood failures directly. +4. A fresh independent context checks the exact final change, unchanged acceptance, + all changed paths and test evidence. The author cannot issue the binding PASS. +5. Report the outcome and checked/not_checked scope, then follow repository delivery + policy. Missing judgment or unchecked acceptance remains NOT_PROVEN. + +No skill installation, workflow-skill loading, bootstrap or ao init is required. +Specialists and evidence persistence are optional. The Validate skill can help +conduct the required review; ao provenance can record explicitly requested proof. +Use ao demo --rpi for the optional full workflow example.`) + return nil +} + +func rpiDemo(w io.Writer) error { fmt.Fprintln(w, `AGENTOPS RPI DEMO 1. Plan refines one active behavior and write scope in the existing intent source. diff --git a/cli/internal/commands/demo/module_test.go b/cli/internal/commands/demo/module_test.go index 25fbb488c..83a7ddb5a 100644 --- a/cli/internal/commands/demo/module_test.go +++ b/cli/internal/commands/demo/module_test.go @@ -34,7 +34,7 @@ func TestModule_CommandAttributes(t *testing.T) { func TestDemoShowsBoundedRepairBoundary(t *testing.T) { var out bytes.Buffer - if err := quickDemo(&out); err != nil { + if err := rpiDemo(&out); err != nil { t.Fatal(err) } for _, want := range []string{"AGENTOPS RPI DEMO", "existing intent source", "runtime derives", "subject-manifest.v1", "fresh validation result", "persistence is optional", "convergence law"} { @@ -54,7 +54,7 @@ func TestDemoConceptsExcludeLifecycleAuthority(t *testing.T) { if err := showConcepts(&out); err != nil { t.Fatal(err) } - for _, want := range []string{"fresh independent judgment", "does not own retries", "Git", "delivery"} { + for _, want := range []string{"independent judgment", "does not own retries", "Git", "delivery", "native coding agent", "optional"} { if !strings.Contains(out.String(), want) { t.Fatalf("concepts missing %q:\n%s", want, out.String()) } @@ -69,8 +69,10 @@ func TestPublishedDemoQuick(t *testing.T) { if err := command.Execute(); err != nil { t.Fatalf("ao demo --quick failed: %v\n%s", err, out.String()) } - if !strings.Contains(out.String(), "AGENTOPS RPI DEMO") { - t.Fatalf("demo output missing the RPI example:\n%s", out.String()) + for _, want := range []string{"AGENTOPS NATIVE DEMO", "ao gate check", "fresh independent context", "binding PASS", "unchanged acceptance", "NOT_PROVEN", "checked/not_checked", "ao demo --rpi"} { + if !strings.Contains(out.String(), want) { + t.Fatalf("demo output missing %q:\n%s", want, out.String()) + } } } diff --git a/cli/internal/commands/init/module.go b/cli/internal/commands/init/module.go index bf88f029f..9e8dc8c7a 100644 --- a/cli/internal/commands/init/module.go +++ b/cli/internal/commands/init/module.go @@ -45,8 +45,11 @@ func (Module) Contract() clicontract.CommandContract { func (m Module) Command() *cobra.Command { return &cobra.Command{ Use: "init", - Short: "Create local AgentOps evidence directories", - Long: `Create the local intent and verdict evidence directories, then add one + Short: "Optionally create local AgentOps evidence directories", + Long: `Optional local evidence setup; native execution needs no init, skills or +bootstrap. Use ao quick-start for the native execution brief. + +Create the local intent and verdict evidence directories, then add one commented, marker-delimited block to this directory's .gitignore (creating the file if it does not exist). The block ignores only machine-local state AgentOps tooling writes: diff --git a/cli/internal/commands/quickstart/module.go b/cli/internal/commands/quickstart/module.go index 38f43fd4c..316c15309 100644 --- a/cli/internal/commands/quickstart/module.go +++ b/cli/internal/commands/quickstart/module.go @@ -1,5 +1,5 @@ // Package quickstart owns Cobra presentation for the `ao quick-start` command. -// The command prints a static RPI workflow summary, so it performs no +// The command prints a static native execution brief, so it performs no // filesystem, process, or clock effect. package quickstart @@ -41,22 +41,35 @@ func (Module) Contract() clicontract.CommandContract { func (Module) Command() *cobra.Command { return &cobra.Command{ Use: "quick-start", - Short: "Show the AgentOps RPI workflow", - Long: `AgentOps is a small semantic evidence layer around agent work. + Short: "Show the native AgentOps execution brief", + Long: `Use your native coding agent and shell to complete the accepted work. +Read the repository's brief and follow its scope, checks and delivery policy. +No skill installation, workflow-skill loading, bootstrap or init is required. -Run the RPI skill for one traversal: - Plan -> Implement -> fresh Validate -> repair to convergence -> report - -The CLI does not claim work, retry, manage Git, or deliver changes. Use -ao gate check for deterministic repository checks and ao provenance for -generic evidence inspection.`, +Use ao gate check for deterministic repository checks. A fresh independent +context judges the exact final change against unchanged acceptance; the +Validate skill is an optional way to conduct that required judgment. +Specialists and machine-readable evidence persistence are optional.`, Args: cobra.NoArgs, GroupID: "start", Run: func(cmd *cobra.Command, _ []string) { - fmt.Fprintln(cmd.OutOrStdout(), "RPI -> Plan -> Implement -> fresh Validate -> repair to convergence -> report") - fmt.Fprintln(cmd.OutOrStdout(), "Deterministic checks: ao gate check") - fmt.Fprintln(cmd.OutOrStdout(), "Semantic judgment: invoke the Validate skill from a fresh context") - fmt.Fprintln(cmd.OutOrStdout(), "Artifact persistence: optional; request verdict.v2 only for machine-readable evidence") + fmt.Fprintln(cmd.OutOrStdout(), `NATIVE AGENTOPS + +1. Read the repository brief; keep accepted behavior and write scope fixed. +2. Use your native agent and shell to implement, run checks and repair known failures. +3. Run required repository checks; ao gate check supplies deterministic facts. +4. Obtain fresh independent judgment of the exact final change against unchanged acceptance. +5. Report the result, checked and not_checked scope, and follow repository delivery policy. + +No skills, bootstrap or ao init are required. Missing independent judgment or +unchecked acceptance means NOT_PROVEN; passing checks alone do not prove completion. +Specialists, including the Validate skill, are optional. From an AgentOps checkout: + ao skills find "your task" # discover an optional specialist + ao skills link --skill security # install only a selected specialist + ao demo --rpi # show the optional full workflow + +Artifact persistence: optional; ao provenance provides explicit evidence tools. +Git, work tracking and delivery remain with the caller and repository.`) }, } } diff --git a/cli/internal/commands/quickstart/module_test.go b/cli/internal/commands/quickstart/module_test.go index 746b1975b..6560281d4 100644 --- a/cli/internal/commands/quickstart/module_test.go +++ b/cli/internal/commands/quickstart/module_test.go @@ -35,20 +35,24 @@ func TestModule_CommandAttributes(t *testing.T) { if command.GroupID != "start" { t.Errorf("GroupID = %q, want start", command.GroupID) } - if command.Short != "Show the AgentOps RPI workflow" { + if command.Short != "Show the native AgentOps execution brief" { t.Errorf("Short = %q", command.Short) } } -func TestCommand_RunPrintsWorkflow(t *testing.T) { +func TestCommand_RunPrintsNativeCompletionBrief(t *testing.T) { command := NewModule().Command() var out bytes.Buffer command.SetOut(&out) command.Run(command, nil) for _, want := range []string{ - "RPI -> Plan -> Implement -> fresh Validate -> repair to convergence -> report", - "Deterministic checks: ao gate check", - "Semantic judgment: invoke the Validate skill from a fresh context", + "native agent and shell", + "ao gate check supplies deterministic facts", + "fresh independent judgment of the exact final change against unchanged acceptance", + "checked and not_checked", + "NOT_PROVEN", + "From an AgentOps checkout", + "including the Validate skill, are optional", "Artifact persistence: optional", } { if !strings.Contains(out.String(), want) { diff --git a/cli/internal/commands/robotdocs/module.go b/cli/internal/commands/robotdocs/module.go index c6f8e0304..0dc5c3648 100644 --- a/cli/internal/commands/robotdocs/module.go +++ b/cli/internal/commands/robotdocs/module.go @@ -48,12 +48,12 @@ func (Module) Command() *cobra.Command { Long: `Print a paste-ready, agent-targeted handbook for the whole ao CLI. The handbook covers the output contract, exit codes, machine-readable -surfaces, and the canonical agent workflow — everything an agent needs to -drive ao without an external documentation lookup.`, +surfaces, and optional AO operations for native execution. Consult it when +an operation is useful; no startup sequence or workflow skills are required.`, GroupID: "core", RunE: func(cmd *cobra.Command, _ []string) error { - fmt.Fprint(cmd.OutOrStdout(), RenderHandbook(cmd.Root())) - return nil + _, err := fmt.Fprint(cmd.OutOrStdout(), RenderHandbook(cmd.Root())) + return err }, } } @@ -65,16 +65,18 @@ func RenderHandbook(root *cobra.Command) string { var b strings.Builder b.WriteString(`# ao — Agent Handbook -ao is the AgentOps CLI: a validation gate plus a provenance record for -agent work — validated output with proof (no verdict = not done). This -handbook is the contract — read it once, then drive ao without guessing. +ao provides deterministic repository checks and explicit evidence operations. +Your native coding agent owns the approach and completes accepted work using +repository instructions and ordinary tools. Consult this handbook on demand; +there is no mandatory AO startup sequence and workflow skills are optional. ## Output contract - stdout is data; stderr is diagnostics. ` + "`ao --json | jq ...`" + ` works without filtering log lines. -- Append ` + "`--json`" + ` (or ` + "`-o json`" + `) to any read-side command for a stable, - parseable structure. ` + "`-o yaml`" + ` and the default ` + "`-o table`" + ` are also available. +- Commands that advertise structured output support ` + "`--json`" + ` or ` + "`-o json`" + `. + Use ` + "`-o yaml`" + ` or ` + "`-o table`" + ` where the command documents support. + This handbook itself is Markdown text. - Output is deterministic where possible: stable ordering, no timestamp leakage into free text. @@ -92,20 +94,38 @@ Diagnostic commands extend this dictionary. Read the precise codes with ## Machine-readable surfaces - ` + "`ao capabilities`" + ` — the full CLI contract as JSON: command surface, - global flags, exit codes, env vars. Run this first. + global flags, exit codes and env vars, when command discovery is useful. - ` + "`ao robot-docs`" + ` — this handbook. - ` + "`ao doctor --robot-triage`" + ` — mega-command: health triage JSON in one call. - ` + "`ao doctor capabilities`" + ` — extended doctor contract (detectors, fixers, exit codes). -## Canonical agent workflow +## Native execution -` + "```" + ` -ao capabilities # discover the contract -ao status --json # where am I, what's initialized -ao doctor --robot-triage # one-call health + remediation -ao gate check --fast --scope head # ordinary deterministic repository checks -` + "```" + ` +Read the repository brief, preserve accepted behavior and scope, implement +with native tools, run required checks and repair known failures. Obtain +fresh independent judgment over the exact final change against unchanged acceptance. +The author cannot issue the binding PASS. Missing judgment or unchecked +acceptance remains NOT_PROVEN; passing deterministic checks alone is insufficient. +Report checked and not_checked scope and follow repository delivery policy. + +The Validate skill is an optional review method; skills, bootstrap and +initialization are not prerequisites. Evidence persistence is optional too. +Git, work tracking and delivery remain with the caller and repository. + +## AO operations on demand + +Choose an operation for a current need; these are independent tools, not steps +to run at the beginning of every task. + +| Current need | Operation | +|--------------|-----------| +| Read the native execution brief | ` + "`ao quick-start`" + ` | +| Inspect command contracts | ` + "`ao capabilities`" + ` | +| Inspect existing durable evidence | ` + "`ao status --json`" + ` | +| Diagnose an AgentOps health problem | ` + "`ao doctor --robot-triage`" + ` | +| Run deterministic repository checks | ` + "`ao gate check`" + ` | +| Inspect explicit evidence operations | ` + "`ao provenance --help`" + ` | ## Environment diff --git a/cli/internal/commands/robotdocs/module_test.go b/cli/internal/commands/robotdocs/module_test.go index c2f53d0e7..ad57114d3 100644 --- a/cli/internal/commands/robotdocs/module_test.go +++ b/cli/internal/commands/robotdocs/module_test.go @@ -3,6 +3,7 @@ package robotdocs import ( "bytes" + "errors" "strings" "testing" @@ -60,7 +61,8 @@ func TestRenderHandbook_ContainsContractSections(t *testing.T) { "## Output contract", "## Exit codes", "## Machine-readable surfaces", - "## Canonical agent workflow", + "## Native execution", + "## AO operations on demand", "## Command surface", "Run `ao --help` for the flags and arguments of any command.", } { @@ -70,6 +72,42 @@ func TestRenderHandbook_ContainsContractSections(t *testing.T) { } } +func TestRenderHandbookKeepsSkillsAndStartupOperationsOptional(t *testing.T) { + out := RenderHandbook(newTestRoot(t)) + for _, want := range []string{ + "native coding agent", + "fresh independent judgment", + "exact final change", + "unchanged acceptance", + "workflow skills are optional", + "NOT_PROVEN", + "ao status --json", + "ao doctor --robot-triage", + } { + if !strings.Contains(out, want) { + t.Errorf("native handbook missing %q", want) + } + } + for _, forbidden := range []string{"read it once", "Run this first", "Canonical agent workflow", "no verdict = not done"} { + if strings.Contains(out, forbidden) { + t.Errorf("optional handbook introduced mandatory workflow %q", forbidden) + } + } +} + +type failedHandbookWriter struct{ err error } + +func (w failedHandbookWriter) Write([]byte) (int, error) { return 0, w.err } + +func TestCommandReportsHandbookOutputFailure(t *testing.T) { + command := NewModule().Command() + want := errors.New("consumer closed output") + command.SetOut(failedHandbookWriter{err: want}) + if err := command.RunE(command, nil); !errors.Is(err, want) { + t.Fatalf("output failure reported as success: %v", err) + } +} + func TestRenderHandbook_ProjectsLiveCommandSurface(t *testing.T) { out := RenderHandbook(newTestRoot(t)) if !strings.Contains(out, "Core:") { diff --git a/cli/internal/commands/skills/link_selection_test.go b/cli/internal/commands/skills/link_selection_test.go new file mode 100644 index 000000000..0e434db97 --- /dev/null +++ b/cli/internal/commands/skills/link_selection_test.go @@ -0,0 +1,111 @@ +package skills + +import ( + "encoding/json" + "os" + "path/filepath" + "reflect" + "strings" + "testing" + + "github.com/boshu2/agentops/cli/internal/skillsapp" +) + +func selectionFixture(t *testing.T) string { + t.Helper() + root := t.TempDir() + for _, dir := range []string{"skills/alpha", "skills/beta", "skills/not-a-skill", "skills-codex"} { + if err := os.MkdirAll(filepath.Join(root, dir), 0o755); err != nil { + t.Fatal(err) + } + } + for _, file := range []string{"registry.json", "PRODUCT.md", "skills/alpha/SKILL.md", "skills/beta/SKILL.md"} { + skillsTestWrite(t, filepath.Join(root, file), "fixture\n") + } + t.Chdir(root) + return root +} + +// Given a source checkout, selecting a specialist installs only that name, +// while the existing command without selectors still installs the full corpus. +func TestLinkSelectionInstallsOnlyNamedSkillsAndPreservesFullInstall(t *testing.T) { + root := selectionFixture(t) + dest := t.TempDir() + for iteration := 0; iteration < 2; iteration++ { + stdout, _, err := execSkills(t, "link", "--dest", dest, "--skill", "beta", "--skill", "beta", "--json") + if err != nil { + t.Fatalf("selected link: %v\n%s", err, stdout) + } + var results []skillsapp.LinkResult + if err := json.Unmarshal([]byte(stdout), &results); err != nil { + t.Fatal(err) + } + if len(results) != 1 { + t.Fatalf("results = %+v", results) + } + names := results[0].Linked + if iteration == 1 { + names = results[0].Present + if len(results[0].Linked) != 0 { + t.Fatalf("repeat linked again: %+v", results) + } + } + if !reflect.DeepEqual(names, []string{"beta"}) { + t.Fatalf("selected names = %v, want [beta]", names) + } + } + if got, err := os.Readlink(filepath.Join(dest, "beta")); err != nil || got != filepath.Join(root, "skills", "beta") { + t.Fatalf("selected target = %q, err %v", got, err) + } + if _, err := os.Lstat(filepath.Join(dest, "alpha")); !os.IsNotExist(err) { + t.Fatalf("unselected skill installed: %v", err) + } + stdout, _, err := execSkills(t, "link", "--dest", dest, "--skill", "beta") + if err != nil || !strings.Contains(stdout, "all requested skills") || strings.Contains(stdout, "all repo skills") { + t.Fatalf("selected repeat claimed full corpus: %v\n%s", err, stdout) + } + stdout, _, err = execSkills(t, "link", "--dest", dest, "--json") + if err != nil { + t.Fatalf("full install: %v\n%s", err, stdout) + } + if _, err := os.Stat(filepath.Join(dest, "alpha", "SKILL.md")); err != nil { + t.Fatalf("full install lost alpha: %v", err) + } +} + +func TestLinkSelectionAcceptsMultipleDistinctNames(t *testing.T) { + selectionFixture(t) + dest := t.TempDir() + stdout, _, err := execSkills(t, "link", "--dest", dest, "--skill", "beta", "--skill", "alpha", "--json") + if err != nil { + t.Fatalf("multiple selected names: %v\n%s", err, stdout) + } + var results []skillsapp.LinkResult + if err := json.Unmarshal([]byte(stdout), &results); err != nil { + t.Fatal(err) + } + if len(results) != 1 || !reflect.DeepEqual(results[0].Linked, []string{"alpha", "beta"}) { + t.Fatalf("multiple selected names = %+v", results) + } +} + +// The whole selection must be checked before even the first valid name can +// create a destination. Invalid names must not silently select the full corpus. +func TestLinkSelectionRejectsInvalidNamesBeforeAnyWrite(t *testing.T) { + for _, name := range []string{"missing", "", " ", ".", "..", "../alpha", "alpha/beta", `alpha\beta`, "/alpha", "alpha,beta", "not-a-skill"} { + t.Run(name, func(t *testing.T) { + selectionFixture(t) + dest := filepath.Join(t.TempDir(), "uncreated") + stdout, stderr, err := execSkills(t, "link", "--dest", dest, "--skill", "alpha", "--skill", name, "--json") + if err == nil { + t.Fatalf("invalid selection %q succeeded: %s", name, stdout) + } + if !strings.Contains(err.Error()+stdout+stderr, "skill") { + t.Fatalf("selection error does not explain correction: %v", err) + } + if _, err := os.Lstat(dest); !os.IsNotExist(err) { + t.Fatalf("invalid selection wrote destination: %v", err) + } + }) + } +} diff --git a/cli/internal/commands/skills/module.go b/cli/internal/commands/skills/module.go index 2bfc49106..9ef92d73b 100644 --- a/cli/internal/commands/skills/module.go +++ b/cli/internal/commands/skills/module.go @@ -53,6 +53,7 @@ type Module struct { // link / unlink linkDest string linkJSON bool + linkSkills []string unlinkDest string unlinkJSON bool } @@ -492,8 +493,14 @@ func (m *Module) linkCommand() *cobra.Command { cmd := &cobra.Command{ Use: "link", Short: "Symlink repo skills into portable and installed runtime skill roots", - Long: `Scan skills/ and create a live-tier symlink for every skill dir that has -no entry yet. By DEFAULT it links into EVERY agent runtime you have installed — + Long: `Optionally install skills from a source checkout. Native execution needs no +skill installation. With no --skill selector, preserve the full install: scan +skills/ and create a live-tier symlink for every skill dir that has no entry yet. +Repeat --skill NAME to install only named specialists. Every name is validated +before any destination is changed; duplicates are installed once. Selection +does not remove existing unselected skills or install dependencies automatically. + +By DEFAULT it links into EVERY agent runtime you have installed — ~/.agents/skills, ~/.claude/skills, ~/.codex/skills, ~/.gemini/skills, ~/.cursor/skills, and ~/.pi/skills — detected by the config root existing under $HOME; --dest overrides to a single dir. Idempotent and non-destructive: skills already @@ -515,6 +522,7 @@ inspect and resolve the named operator-owned path explicitly. ao skills link # link missing into every installed runtime ao skills link --dry-run # show what's missing without linking + ao skills link --skill security --skill test # install only these specialists git pull && ao skills link # track main: pick up newly-landed skills ao skills link --dest ~/.codex/skills # link into ONE specific dir only`, Args: cobra.NoArgs, @@ -522,6 +530,7 @@ inspect and resolve the named operator-owned path explicitly. } cmd.Flags().StringVar(&m.linkDest, "dest", "", "Link into this single dir instead of the auto-detected roots (default: ~/.agents plus every installed runtime)") cmd.Flags().BoolVar(&m.linkJSON, "json", false, "Emit machine-readable JSON") + cmd.Flags().StringArrayVar(&m.linkSkills, "skill", nil, "Install only this skill; repeat for more names (default: all skills)") return cmd } @@ -538,7 +547,11 @@ func (m *Module) runLink(cmd *cobra.Command, _ []string) error { return err } - results, anyErr := skillsapp.LinkAllDests(skillsDir, dests, m.host.DryRun()) + results, anyErr, err := skillsapp.LinkAllDests(skillsDir, dests, m.host.DryRun(), m.linkSkills...) + if err != nil { + cmd.SilenceUsage = true + return err + } if m.linkJSON { enc := json.NewEncoder(cmd.OutOrStdout()) diff --git a/cli/internal/evidence/judgment.go b/cli/internal/evidence/judgment.go index 4e1365c42..3b1d6524c 100644 --- a/cli/internal/evidence/judgment.go +++ b/cli/internal/evidence/judgment.go @@ -28,6 +28,9 @@ func VerifyJudgments(o JudgmentOptions) (*JudgmentResult, error) { if err != nil { return nil, err } + if len(manifest.Entries) == 0 { + return nil, fmt.Errorf("subject manifest has no entries; nonempty implementation candidate required") + } var base *Manifest if o.BaseManifest != "" { base, err = LoadManifest(o.BaseManifest) @@ -42,7 +45,8 @@ func VerifyJudgments(o JudgmentOptions) (*JudgmentResult, error) { if err != nil { return nil, err } - result := &JudgmentResult{Satisfied: true, SubjectManifestDigest: manifest.Digest, AcceptanceDigest: Hash(intent), Legs: []JudgmentLeg{}, Problems: []string{}} + result := &JudgmentResult{Satisfied: true, SubjectManifestDigest: manifest.Digest, AcceptanceDigest: Hash(intent), + RequiredCriteria: append([]string(nil), o.RequiredCriteria...), Legs: []JudgmentLeg{}, Problems: []string{}} state := judgmentState{options: o, root: root, result: result, seen: map[string]bool{}, contexts: map[string]string{}} for _, path := range o.Verdicts { state.inspectVerdict(path) @@ -57,6 +61,9 @@ func VerifyJudgments(o JudgmentOptions) (*JudgmentResult, error) { } func checkJudgePolicy(o JudgmentOptions) error { + if err := checkCriterionPolicy(o.RequiredCriteria); err != nil { + return err + } if len(o.Required) == 0 || !knownJudgeIdentity(o.AuthorContextID) { return fmt.Errorf("required profiles and independent author identity must be nonempty") } @@ -140,6 +147,7 @@ func (s *judgmentState) inspectLeg(path, ref string, v *verdictcheck.Verdict, r leg.Problems = append(leg.Problems, "requested_profile_mismatch") } leg.Problems = append(leg.Problems, receiptBindingProblems(s.options, s.result, v, r)...) + leg.Problems = append(leg.Problems, criterionCoverageProblems(s.options.RequiredCriteria, v.Criteria)...) // Runtime/family identity is chosen by the independent profile. A receipt // cannot route parsing or retrieval to a substituted/disallowed provider. if r.Requested.Runtime == profile.Runtime && r.Requested.Family == profile.Family { @@ -152,6 +160,51 @@ func (s *judgmentState) inspectLeg(path, ref string, v *verdictcheck.Verdict, r s.result.Legs = append(s.result.Legs, leg) } +func checkCriterionPolicy(required []string) error { + if required == nil { + return nil + } + if len(required) == 0 { + return fmt.Errorf("selected required criterion IDs must be nonempty") + } + seen := map[string]bool{} + for _, id := range required { + if strings.TrimSpace(id) == "" || id != strings.TrimSpace(id) || seen[id] { + return fmt.Errorf("required criterion IDs must be nonempty, unpadded and unique") + } + seen[id] = true + } + return nil +} + +// Criterion IDs establish exact coverage of the caller's selected set. The +// fresh reviewer still owns each ID's mapping to intent and supporting evidence. +func criterionCoverageProblems(required []string, criteria []verdictcheck.Criterion) []string { + if required == nil { + return nil + } + allowed, seen := map[string]bool{}, map[string]int{} + for _, id := range required { + allowed[id] = true + } + problems := []string{} + for _, criterion := range criteria { + seen[criterion.ID]++ + if seen[criterion.ID] == 2 { + problems = append(problems, "duplicate_criterion: "+criterion.ID) + } + if seen[criterion.ID] == 1 && !allowed[criterion.ID] { + problems = append(problems, "unexpected_criterion: "+criterion.ID) + } + } + for _, id := range required { + if seen[id] == 0 { + problems = append(problems, "missing_required_criterion: "+id) + } + } + return problems +} + func receiptBindingProblems(o JudgmentOptions, result *JudgmentResult, v *verdictcheck.Verdict, r *JudgmentReceipt) []string { problems := []string{} if r.SubjectManifestDigest != result.SubjectManifestDigest || v.SubjectManifestDigest != result.SubjectManifestDigest { diff --git a/cli/internal/evidence/judgment_criteria_test.go b/cli/internal/evidence/judgment_criteria_test.go new file mode 100644 index 000000000..e9ca9fab5 --- /dev/null +++ b/cli/internal/evidence/judgment_criteria_test.go @@ -0,0 +1,90 @@ +package evidence + +import ( + "os" + "slices" + "strings" + "testing" +) + +func TestJudgmentsRejectPartialRequiredCriterionCoverage(t *testing.T) { + f := newJudgmentFixture(t, "codex") + intent := []byte("c1: the value is correct\nc2: changes stay in scope\n") + write(t, f.options.Intent, intent) + f.receipt.AcceptanceDigest = Hash(intent) + f.options.RequiredCriteria = []string{"c1", "c2"} + f.publish(t, nil) // A valid, correctly bound PASS contains only c1. + result, err := VerifyJudgments(f.options) + if err != nil { + t.Fatal(err) + } + if result.Satisfied || len(result.Legs) != 1 || result.Legs[0].Verdict != "PASS" || !strings.Contains(strings.Join(result.Legs[0].Problems, ";"), "missing_required_criterion: c2") { + t.Fatalf("partial PASS must remain visible and unproven: %+v", result) + } +} + +func TestJudgmentsRequireExactSelectedCriterionIDs(t *testing.T) { + cases := []struct { + name string + required, actual []string + verdict string + wantSatisfied bool + wantProblem string + }{ + {"complete PASS", []string{"c1", "c2"}, []string{"c1", "c2"}, "PASS", true, ""}, + {"order independent", []string{"c1", "c2"}, []string{"c2", "c1"}, "PASS", true, ""}, + {"partial PASS", []string{"c1", "c2"}, []string{"c1"}, "PASS", false, "missing_required_criterion: c2"}, + {"duplicate actual", []string{"c1", "c2"}, []string{"c1", "c2", "c1"}, "PASS", false, "duplicate_criterion: c1"}, + {"unknown actual", []string{"c1", "c2"}, []string{"c1", "c2", "other"}, "PASS", false, "unexpected_criterion: other"}, + {"unknown expected", []string{"c1", "other"}, []string{"c1", "c2"}, "PASS", false, "missing_required_criterion: other"}, + {"complete FAIL", []string{"c1", "c2"}, []string{"c1", "c2"}, "FAIL", false, "verdict_not_pass"}, + {"partial FAIL", []string{"c1", "c2"}, []string{"c1"}, "FAIL", false, "missing_required_criterion: c2"}, + {"legacy unselected policy", nil, []string{"c1"}, "PASS", true, ""}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + f := newJudgmentFixture(t, "codex") + f.options.RequiredCriteria = tc.required + f.publish(t, func(v map[string]any) { + v["verdict"] = tc.verdict + criteria := []any{} + for _, id := range tc.actual { + criteria = append(criteria, map[string]any{"id": id, "result": tc.verdict, "evidence_refs": []string{"test:receipt"}}) + } + v["criteria"] = criteria + }) + before, err := os.ReadFile(f.options.Verdicts[0]) + if err != nil { + t.Fatal(err) + } + result, err := VerifyJudgments(f.options) + if err != nil { + t.Fatal(err) + } + if result.Satisfied != tc.wantSatisfied || !slices.Equal(result.RequiredCriteria, tc.required) || len(result.Legs) != 1 || result.Legs[0].Verdict != tc.verdict { + t.Fatalf("coverage or original verdict changed: %+v", result) + } + if tc.wantProblem != "" && !strings.Contains(strings.Join(result.Legs[0].Problems, ";"), tc.wantProblem) { + t.Fatalf("wanted %s, got %+v", tc.wantProblem, result) + } + if tc.name == "complete FAIL" && !slices.Equal(result.Legs[0].Problems, []string{"verdict_not_pass"}) { + t.Fatalf("complete FAIL gained unrelated problems: %+v", result) + } + after, err := os.ReadFile(f.options.Verdicts[0]) + if err != nil || string(before) != string(after) { + t.Fatal("original verdict mutated") + } + }) + } +} + +func TestJudgmentsInvalidCriterionPolicyBeforeSubjectReads(t *testing.T) { + for _, required := range [][]string{{}, {""}, {" "}, {"c1", "c1"}, {" c1"}} { + f := newJudgmentFixture(t, "codex") + f.options.RequiredCriteria = required + f.options.Manifest = "/missing-subject-must-not-be-read" + if result, err := VerifyJudgments(f.options); err == nil || !strings.Contains(err.Error(), "criterion IDs") || result != nil { + t.Fatalf("invalid policy %q did not fail before reads: %+v, %v", required, result, err) + } + } +} diff --git a/cli/internal/evidence/judgment_test.go b/cli/internal/evidence/judgment_test.go index 15c56e575..b09d5d976 100644 --- a/cli/internal/evidence/judgment_test.go +++ b/cli/internal/evidence/judgment_test.go @@ -83,6 +83,55 @@ func TestJudgmentsNativeRequiredLegs(t *testing.T) { } } +func TestJudgmentsRejectEmptyImplementationSubject(t *testing.T) { + f := newJudgmentFixture(t, "codex") + empty := t.TempDir() + manifest, err := BuildManifest(empty, []string{"."}, nil, nil, nil) + if err != nil { + t.Fatal(err) + } + if len(manifest.Entries) != 0 { + t.Fatal("fixture must describe an empty implementation") + } + f.options.Root = empty + writeJSON(t, f.options.Manifest, manifest) + f.receipt.SubjectManifestDigest = manifest.Digest + f.publish(t, nil) + + result, err := VerifyJudgments(f.options) + if err == nil || !strings.Contains(err.Error(), "subject manifest has no entries") { + t.Fatalf("empty implementation must not satisfy a judgment: result %+v, error %v", result, err) + } +} + +func TestJudgmentsAcceptDeletionOnlySubject(t *testing.T) { + f := newJudgmentFixture(t, "codex") + base, err := LoadManifest(f.options.Manifest) + if err != nil { + t.Fatal(err) + } + f.options.BaseManifest = filepath.Join(f.out, "base-manifest.json") + writeJSON(t, f.options.BaseManifest, base) + if err = os.Remove(filepath.Join(f.options.Root, "value")); err != nil { + t.Fatal(err) + } + manifest, err := BuildManifest(f.options.Root, []string{"value"}, nil, base, nil) + if err != nil { + t.Fatal(err) + } + if len(manifest.Entries) != 1 || manifest.Entries[0].Kind != "deletion" { + t.Fatalf("expected one real deletion: %+v", manifest.Entries) + } + writeJSON(t, f.options.Manifest, manifest) + f.receipt.SubjectManifestDigest = manifest.Digest + f.publish(t, nil) + + result, err := VerifyJudgments(f.options) + if err != nil || !result.Satisfied { + t.Fatalf("a verified deletion is a nonempty subject: result %+v, error %v", result, err) + } +} + func TestJudgmentsRequiredFailures(t *testing.T) { tests := []struct { name string diff --git a/cli/internal/evidence/judgment_types.go b/cli/internal/evidence/judgment_types.go index 896447624..73aa99919 100644 --- a/cli/internal/evidence/judgment_types.go +++ b/cli/internal/evidence/judgment_types.go @@ -45,13 +45,17 @@ type JudgmentReceipt struct { // The helper is a local reader: it never transmits bytes or authorizes a model // launch. The invoking runtime must enforce disclosure policy before dispatch. type JudgmentOptions struct { - Root string - EvidenceRoot string - Manifest string - BaseManifest string - Intent string - AuthorContextID string - Required []JudgeProfile + Root string + EvidenceRoot string + Manifest string + BaseManifest string + Intent string + AuthorContextID string + Required []JudgeProfile + // RequiredCriteria is the caller's complete acceptance ID set. Nil retains + // legacy provenance behavior; a selected set must be nonempty and unique. + // IDs are never derived from candidate verdicts or parsed from intent prose. + RequiredCriteria []string AllowedProviders []string Verdicts []string } @@ -72,6 +76,7 @@ type JudgmentResult struct { Satisfied bool `json:"satisfied"` SubjectManifestDigest string `json:"subject_manifest_digest"` AcceptanceDigest string `json:"acceptance_digest"` + RequiredCriteria []string `json:"required_criteria,omitempty"` Legs []JudgmentLeg `json:"legs"` Problems []string `json:"problems"` } diff --git a/cli/internal/skillsapp/link.go b/cli/internal/skillsapp/link.go index 755872afe..2afb97f6a 100644 --- a/cli/internal/skillsapp/link.go +++ b/cli/internal/skillsapp/link.go @@ -6,6 +6,7 @@ import ( "io" "os" "path/filepath" + "slices" "sort" "strings" ) @@ -33,7 +34,7 @@ type LinkResult struct { // // Repairing an existing wrong or broken symlink is deliberately out of scope; // it is reported as a conflict so an operator can resolve ownership explicitly. -func linkMissingSkills(srcDir, destDir string, dryRun bool) (LinkResult, error) { +func linkMissingSkills(srcDir, destDir string, dryRun bool, selected ...string) (LinkResult, error) { res := LinkResult{Dest: destDir, DryRun: dryRun} // Fail-closed on an unresolved source: an empty srcDir would let @@ -57,6 +58,9 @@ func linkMissingSkills(srcDir, destDir string, dryRun bool) (LinkResult, error) continue } name := e.Name() + if len(selected) > 0 && !slices.Contains(selected, name) { + continue + } src := filepath.Join(absSrc, name) if _, statErr := os.Stat(filepath.Join(src, "SKILL.md")); statErr != nil { continue // not a skill dir — no SKILL.md @@ -103,23 +107,59 @@ func linkMissingSkills(srcDir, destDir string, dryRun bool) (LinkResult, error) return res, nil } -// LinkAllDests links the repo skills into every destination, RESILIENTLY: a +// validateSelection checks the complete selection before any destination is +// touched. Names are exact source directory names, not paths or comma lists. +// No selectors preserves the explicit full-install behavior. +func validateSelection(srcDir string, names []string) ([]string, error) { + if len(names) == 0 { + return nil, nil + } + if strings.TrimSpace(srcDir) == "" { + return nil, fmt.Errorf("skills source dir is empty") + } + selected := make([]string, 0, len(names)) + for _, name := range names { + if name == "" || strings.TrimSpace(name) != name || name == "." || name == ".." || strings.ContainsAny(name, `/\,`) { + return nil, fmt.Errorf("invalid skill name %q: use one exact skill directory name per --skill", name) + } + info, err := os.Lstat(filepath.Join(srcDir, name)) + if err != nil || !info.IsDir() { + return nil, fmt.Errorf("unknown skill %q: select a directory in the repository skills/ tree", name) + } + info, err = os.Stat(filepath.Join(srcDir, name, "SKILL.md")) + if err != nil || !info.Mode().IsRegular() { + return nil, fmt.Errorf("invalid skill %q: SKILL.md must be a regular file", name) + } + if !slices.Contains(selected, name) { + selected = append(selected, name) + } + } + sort.Strings(selected) + return selected, nil +} + +// LinkAllDests links selected repo skills into every destination, RESILIENTLY: a // per-dest error is captured on that dest's result and the fan-out continues to // the remaining runtimes rather than aborting (which would leave earlier dests // mutated and later ones silently skipped). Returns the per-dest results and -// whether any dest errored. -func LinkAllDests(srcDir string, dests []string, dryRun bool) ([]LinkResult, bool) { +// whether any dest errored. A selection error returns before any write; an +// empty selection preserves full installation. +func LinkAllDests(srcDir string, dests []string, dryRun bool, names ...string) ([]LinkResult, bool, error) { + selected, err := validateSelection(srcDir, names) + if err != nil { + return nil, false, err + } results := make([]LinkResult, 0, len(dests)) anyErr := false for _, dest := range dests { - res, err := linkMissingSkills(srcDir, dest, dryRun) // nosemgrep -- res is a value struct (never nil); setting res.Err on error cannot nil-deref. + res, err := linkMissingSkills(srcDir, dest, dryRun, selected...) // nosemgrep -- res is a value struct (never nil); setting res.Err on error cannot nil-deref. if err != nil { res.Err = err.Error() anyErr = true } results = append(results, res) } - return results, anyErr + return results, anyErr, nil } // RenderLinkResult prints one destination's link summary. @@ -147,6 +187,6 @@ func RenderLinkResult(out io.Writer, res LinkResult) { fmt.Fprintf(out, " ! %s (real dir — foreign corpus, not clobbered)\n", n) } if len(res.Linked) == 0 && len(res.Conflicts) == 0 { - fmt.Fprintln(out, " all repo skills already live-linked.") + fmt.Fprintln(out, " all requested skills already live-linked.") } } diff --git a/cli/internal/skillsapp/link_test.go b/cli/internal/skillsapp/link_test.go index 974befa7d..da0c3df79 100644 --- a/cli/internal/skillsapp/link_test.go +++ b/cli/internal/skillsapp/link_test.go @@ -285,7 +285,10 @@ func TestLinkAllDests_ResilientAcrossDests(t *testing.T) { } bad := filepath.Join(badParent, "skills") - results, anyErr := LinkAllDests(src, []string{bad, good}, false) + results, anyErr, err := LinkAllDests(src, []string{bad, good}, false) + if err != nil { + t.Fatal(err) + } if !anyErr { t.Fatal("anyErr should be true when a dest fails") @@ -333,3 +336,79 @@ func TestRenderLinkResult_ErrorDryRunAndConflictLines(t *testing.T) { t.Errorf("conflict line must state the non-clobber guarantee, got:\n%s", out) } } + +func TestSelectedLinkDryRunPreflightsAllNamesAcrossDestinations(t *testing.T) { + src := t.TempDir() + mkSkill(t, src, "alpha") + mkSkill(t, src, "beta") + dests := []string{filepath.Join(t.TempDir(), "one"), filepath.Join(t.TempDir(), "two")} + for _, dryRun := range []bool{false, true} { + results, _, err := LinkAllDests(src, dests, dryRun, "alpha", "missing") + if err == nil || len(results) != 0 { + t.Fatalf("invalid selection attempted destinations: %+v, %v", results, err) + } + } + results, anyErr, err := LinkAllDests(src, dests, true, "beta", "beta") + if err != nil || anyErr || len(results) != 2 { + t.Fatalf("selected dry-run: %+v, %v, %v", results, anyErr, err) + } + for _, res := range results { + if !res.DryRun || len(res.Linked) != 1 || res.Linked[0] != "beta" { + t.Fatalf("wrong selected dry-run: %+v", res) + } + } + for _, dest := range dests { + if _, err := os.Lstat(dest); !os.IsNotExist(err) { + t.Fatalf("preflight or dry-run created %s: %v", dest, err) + } + } +} + +func TestSelectedLinkPreservesForeignAndUnselectedEntries(t *testing.T) { + src, dest := t.TempDir(), t.TempDir() + for _, name := range []string{"alpha", "beta", "gamma"} { + mkSkill(t, src, name) + } + foreign := filepath.Join(dest, "alpha") + if err := os.WriteFile(foreign, []byte("user-owned"), 0o644); err != nil { + t.Fatal(err) + } + unselected := filepath.Join(dest, "gamma") + if err := os.Symlink(filepath.Join(src, "gamma"), unselected); err != nil { + t.Fatal(err) + } + results, anyErr, err := LinkAllDests(src, []string{dest}, false, "beta", "alpha") + if err != nil || anyErr || len(results) != 1 { + t.Fatalf("selected link: %+v, %v, %v", results, anyErr, err) + } + res := results[0] + if len(res.Conflicts) != 1 || res.Conflicts[0] != "alpha" || len(res.Linked) != 1 || res.Linked[0] != "beta" || len(res.Present) != 0 { + t.Fatalf("selection escaped or lost ownership: %+v", res) + } + if data, err := os.ReadFile(foreign); err != nil || string(data) != "user-owned" { + t.Fatalf("foreign entry changed: %q, %v", data, err) + } + if target, err := os.Readlink(unselected); err != nil || target != filepath.Join(src, "gamma") { + t.Fatalf("unselected existing entry changed: %q, %v", target, err) + } +} + +func TestSelectedLinkContinuesAfterDestinationError(t *testing.T) { + src, dest := t.TempDir(), t.TempDir() + mkSkill(t, src, "alpha") + mkSkill(t, src, "beta") + bad := filepath.Join(t.TempDir(), "file") + if err := os.WriteFile(bad, []byte("user-owned"), 0o644); err != nil { + t.Fatal(err) + } + results, anyErr, err := LinkAllDests(src, []string{bad, dest}, false, "alpha") + if err != nil || !anyErr || len(results) != 2 || results[0].Err == "" || results[1].Err != "" { + t.Fatalf("selected destination error handling: %+v, %v, %v", results, anyErr, err) + } + if _, err := os.Stat(filepath.Join(dest, "alpha", "SKILL.md")); err != nil { + t.Fatalf("good destination skipped: %v", err) + } + if _, err := os.Lstat(filepath.Join(dest, "beta")); !os.IsNotExist(err) { + t.Fatalf("unselected name leaked: %v", err) + } +} diff --git a/cli/internal/skilltrial/report.go b/cli/internal/skilltrial/report.go index e81efb041..dab756822 100644 --- a/cli/internal/skilltrial/report.go +++ b/cli/internal/skilltrial/report.go @@ -93,6 +93,7 @@ type Report struct { RewardKey string `json:"reward_key"` Jobs []Job `json:"jobs"` Sessions []Session `json:"sessions"` + Work *Work `json:"work"` Limits []string `json:"limits"` } diff --git a/cli/internal/skilltrial/work.go b/cli/internal/skilltrial/work.go new file mode 100644 index 000000000..179269797 --- /dev/null +++ b/cli/internal/skilltrial/work.go @@ -0,0 +1,172 @@ +package skilltrial + +import ( + "bytes" + "encoding/json" + "sort" + + "github.com/boshu2/agentops/cli/internal/evidence" + "github.com/boshu2/agentops/cli/internal/parser" +) + +// Work joins one caller-selected subject to an observed native author. Status +// reports supplied independent judgment, not a new semantic verdict. Endpoint +// reward and execution remain separate; no trial is selected from retries. +type Work struct { + Status string `json:"status"` + Execution string `json:"execution"` + Association string `json:"association"` + AuthorContextID string `json:"author_context_id"` + TrialDirectory string `json:"trial_directory"` + ManifestPath string `json:"manifest_path"` + IntentPath string `json:"intent_path"` + AuthorEvidence []Evidence `json:"author_evidence"` + Judgments *evidence.JudgmentResult `json:"judgments"` + Problems []string `json:"problems"` +} + +// InspectWork reuses the provenance owner to inspect original verdicts, exact +// subject/acceptance and native reviewer evidence. An inspection error remains +// visible in the report instead of hiding unsuccessful or incomplete attempts. +func InspectWork(report *Report, options evidence.JudgmentOptions) *Work { + w := &Work{Status: "not_proven", Execution: "unknown", Association: "missing", AuthorContextID: options.AuthorContextID, + ManifestPath: options.Manifest, IntentPath: options.Intent, AuthorEvidence: []Evidence{}, Problems: []string{}} + w.associate(report) + result, err := evidence.VerifyJudgments(options) + w.Judgments = result + if len(options.RequiredCriteria) == 0 { + w.Problems = append(w.Problems, "required_criteria_missing") + } + if err != nil { + w.Problems = append(w.Problems, "judgment_verification: "+err.Error()) + } + if len(w.Problems) == 0 && result != nil { + if result.Satisfied && w.Execution == "completed" { + w.Status = "accepted" + } else if verifiedFailure(result) { + w.Status = "failed" + } + } + if w.Execution != "completed" { + w.Problems = append(w.Problems, "author_execution_"+w.Execution) + } + sort.Strings(w.Problems) + return w +} + +// Non-PASS is deliberately unsatisfied in VerifyJudgments. A FAIL whose only +// leg problem is verdict_not_pass still supplies an independently bound failure. +// Other missing or invalid legs remain unchanged in the original result. +func verifiedFailure(result *evidence.JudgmentResult) bool { + for _, leg := range result.Legs { + if leg.Verdict == "FAIL" && len(leg.Problems) == 1 && leg.Problems[0] == "verdict_not_pass" { + return true + } + } + return false +} + +func (w *Work) associate(report *Report) { + var author *Session + for i := range report.Sessions { + if report.Sessions[i].ID == w.AuthorContextID { + author = &report.Sessions[i] + break + } + } + if author == nil || len(author.Copies) == 0 { + w.Problems = append(w.Problems, "author_session_missing") + return + } + w.Association = "standalone_session" + if len(author.Diagnostics) > 0 { + w.Problems = append(w.Problems, "author_session_diagnostics") + } + trials, hashes := map[string]bool{}, map[string]bool{} + for _, copy := range author.Copies { + w.AuthorEvidence = append(w.AuthorEvidence, copy.Evidence) + hashes[copy.Evidence.SHA256] = true + if copy.Trial != "" { + trials[copy.Trial] = true + } + if authorAccountingUnverified(copy.Accounting) { + w.Problems = append(w.Problems, "author_accounting_unverified") + } + } + if len(hashes) != 1 || len(trials) > 1 { + w.Association = "ambiguous" + w.Problems = append(w.Problems, "author_session_copies_or_trial_assignment_ambiguous") + return + } + w.Execution = authorExecution(author.Copies[0].Accounting) + for directory := range trials { + w.Association, w.TrialDirectory = "trial", directory + for _, job := range report.Jobs { + for _, trial := range job.Trials { + if trial.Directory == directory { + w.bindTrialExecution(trial) + return + } + } + } + w.Problems = append(w.Problems, "associated_trial_missing") + } +} + +func authorAccountingUnverified(accounting *parser.CodexAccounting) bool { + if accounting == nil { + return true + } + for _, diagnostic := range accounting.Diagnostics { + // This existing parser diagnostic concerns aggregate usage only. Keep it + // in native accounting without invalidating independently observed author + // identity or execution. Unknown and parse diagnostics still fail closed. + if diagnostic.Message != "cumulative counters decreased; last native snapshot retained, aggregation requires review" { + return true + } + } + return false +} + +func authorExecution(accounting *parser.CodexAccounting) string { + if accounting == nil { + return "unknown" + } + // These terminal markers are retained by accounting as uninterpreted event + // counts. Their order relative to its latest supported marker is unavailable; + // do not let an earlier task_complete stand in for proven completion. + for _, marker := range []string{"event_msg/turn_aborted", "event_msg/error"} { + if accounting.Uninterpreted[marker] > 0 { + return "unknown" + } + } + var payload struct { + Error json.RawMessage `json:"error"` + } + if json.Unmarshal(accounting.LatestTurnPayload, &payload) == nil && len(payload.Error) > 0 && !bytes.Equal(bytes.TrimSpace(payload.Error), []byte("null")) { + return "failed" + } + switch accounting.LatestTurnState { + case "task_complete": + return "completed" + case "task_started": + return "unfinished" + case "task_aborted": + return "failed" + default: + return "unknown" + } +} + +func (w *Work) bindTrialExecution(trial Trial) { + switch trial.Outcome { + case "execution_error", "infrastructure_error": + w.Execution = "failed" + case "created", "unfinished": + w.Execution = "unfinished" + default: + if !trial.Completed { + w.Execution = "unknown" + } + } +} diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 589aa0228..3980ea624 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -1,18 +1,20 @@ # Architecture -AgentOps has a small semantic core and optional adapters around it. +AgentOps defaults to native execution with zero mandatory skills. Its small +semantic core binds acceptance to exact content and fresh independent judgment; +optional skills and adapters package guidance around that boundary. ```text existing bead or caller intent -> on-demand planning and bounded implementation -> runtime-derived subject-manifest.v1 + check receipts - -> fresh Validate + -> fresh independent judgment -> PASS | FAIL | NOT_PROVEN -> direct repair and fresh revalidation within real bounds when needed -> completed acceptance or truthful unfinished result ``` -## Core +## Optional workflow guidance - **Plan** shapes missing intent and revises disproved approaches under unchanged acceptance and scope; clear work need not invoke it. @@ -33,7 +35,7 @@ genuine causal stall admits at most one bounded fresh helper; an unhelpful answer, cancellation, refusal or a spent real bound ends the attempt. Explicit repair-round bounds still apply, and no invocation renews an allowance. The [fixed-dispatch adapter](../skills/rpi/references/bounded-adapter.md) keeps its -narrower optional contract; it does not govern native RPI execution. +narrower optional contract; it does not govern direct native execution. ## Hexagonal boundary diff --git a/docs/CONTRIBUTING.md b/docs/CONTRIBUTING.md index cbe4931c7..49677dbe5 100644 --- a/docs/CONTRIBUTING.md +++ b/docs/CONTRIBUTING.md @@ -104,7 +104,7 @@ scripts/docs-build.sh --serve MkDocs-specific expectations: -- Every internal link must resolve. `mkdocs build --strict` fails on unresolved relative links; `tests/docs/validate-links.sh` catches the same class without a Python toolchain. `--check` tolerates only the enumerated, intentional cross-references in `tests/docs/mkdocs-strict-allowlist.txt` (docs pages linking to real repo artifacts outside the site) — see [Docs-Build Dispositions](docs-build-dispositions.md); any *other* strict warning fails the build, so fix the link rather than silencing it. +- Every internal link must resolve. `mkdocs build --strict` fails on unresolved relative links; `tests/docs/validate-links.sh` catches the same class without a Python toolchain. `--check` tolerates only the enumerated, intentional cross-references in `tests/docs/mkdocs-strict-allowlist.txt` (docs pages linking to real repo artifacts outside the site) — see [the strict-allowlist policy](../tests/docs/mkdocs-strict-allowlist.txt); any *other* strict warning fails the build, so fix the link rather than silencing it. - Skill pages and the CLI reference are **generated at build time** from `skills/**/SKILL.md` and `cli/docs/COMMANDS.md` respectively — do not hand-author `docs/skills/*.md` or `docs/cli/commands.md`. - Navigation is declared in `mkdocs.yml` under `nav:`. New top-level docs need an entry there. diff --git a/docs/MIGRATION.md b/docs/MIGRATION.md index 9331ea7bd..bcdf0a7b6 100644 --- a/docs/MIGRATION.md +++ b/docs/MIGRATION.md @@ -3,7 +3,7 @@ AgentOps now owns one small product boundary: ```text -RPI charter -> Plan if needed -> Implement and check -> fresh Validate -> finish +Accepted intent -> native implementation and checks -> fresh independent judgment -> finish ``` (The 3.0 through 3.6 releases stopped after one validation; ADR-0017 added the @@ -12,7 +12,19 @@ bounded repair phase.) Known defects can be repaired directly and an approach can change within the accepted outcome, scope and real allowance. The caller owns changes to that acceptance, new allowances, scheduling, trackers, Git and delivery. Deterministic repository checks stay -under `ao gate check`; semantic judgment is the Validate skill. +under `ao gate check`; semantic judgment belongs to a fresh reviewer. + +## Native default — 2026-09-10 + +The default requires zero AgentOps skills. `ao quick-start` and `ao demo` give +read-only native guidance. `ao demo --rpi` retains the explicit workflow example. +`ao init` remains optional evidence setup. No session bootstrap is required. + +The skill sources, explicit full plugins and no-selector `ao skills link` are +preserved. From a checkout, repeat `--skill NAME` to link selected skills only. +Selection does not uninstall existing links or rewrite user configuration. +The optional RPI and Validate skills retain their contracts; native work uses +the same exact-content, author-distinct judgment bar without invoking them. ## Removed responsibilities diff --git a/docs/adr/ADR-0001-ddd-hexagonal-adoption.md b/docs/adr/ADR-0001-ddd-hexagonal-adoption.md index 656fa3cb2..5db9e1bbc 100644 --- a/docs/adr/ADR-0001-ddd-hexagonal-adoption.md +++ b/docs/adr/ADR-0001-ddd-hexagonal-adoption.md @@ -20,7 +20,7 @@ rationale for the retained architecture, not permission to rebuild the packet. ## Historical context -[`docs/cdlc.md`](../cdlc.md), [`PRODUCT.md`](https://github.com/boshu2/agentops/blob/main/PRODUCT.md), and the `ddd-bounded-context` / `hexagonal-architecture` rows in [`PRACTICE-REGISTRY.md`](https://github.com/boshu2/agentops/blob/main/PRACTICE-REGISTRY.md) already identify DDD + Hexagonal as the load-bearing architectural style. The encoding existed implicitly in `skills/rpi/references/phase-data-contracts.md` (linked-intent packet), `skills/domain/` (ubiquitous language), and the practice slug registry — but it was not mechanically enforced. This left the architecture as documentation rather than code-as-truth. +`docs/cdlc.md` (historical reference; retired path: `../cdlc.md`), [`PRODUCT.md`](https://github.com/boshu2/agentops/blob/main/PRODUCT.md), and the `ddd-bounded-context` / `hexagonal-architecture` rows in [`PRACTICE-REGISTRY.md`](https://github.com/boshu2/agentops/blob/main/PRACTICE-REGISTRY.md) already identify DDD + Hexagonal as the load-bearing architectural style. The encoding existed implicitly in `skills/rpi/references/phase-data-contracts.md` (linked-intent packet), `skills/domain/` (ubiquitous language), and the practice slug registry — but it was not mechanically enforced. This left the architecture as documentation rather than code-as-truth. ## Decision @@ -52,6 +52,6 @@ Adopt DDD + Hexagonal as encoded architecture, starting with `ExecutionPacket` a - Eric Evans, 2003. *Domain-Driven Design*. - Bertrand Meyer. *Object-Oriented Software Construction* — Design by Contract. - [`PRACTICE-REGISTRY.md`](https://github.com/boshu2/agentops/blob/main/PRACTICE-REGISTRY.md) — canonical practice slugs for `ddd-bounded-context`, `hexagonal-architecture`, and `adr`. -- [`docs/cdlc.md`](../cdlc.md) — CDLC doctrine and narrow-waist framing. +- `docs/cdlc.md` (historical reference; retired path: `../cdlc.md`) — CDLC doctrine and narrow-waist framing. - [Ports and Adapters](../architecture/ports-and-adapters.md) — this ADR's companion overview. - [Context Map](../contracts/context-map.md) — auto-generated map of bounded contexts. diff --git a/docs/adr/ADR-0002-agentops-3-hookless-cdlc-rearchitecture.md b/docs/adr/ADR-0002-agentops-3-hookless-cdlc-rearchitecture.md index c7c330491..4ca4f9d70 100644 --- a/docs/adr/ADR-0002-agentops-3-hookless-cdlc-rearchitecture.md +++ b/docs/adr/ADR-0002-agentops-3-hookless-cdlc-rearchitecture.md @@ -107,7 +107,7 @@ This ADR is accepted when: ## References -- [CDLC](../cdlc.md) +- CDLC (historical reference; retired path: `../cdlc.md`) - [Operating Loop](../architecture/operating-loop.md) - [Ports and Adapters](../architecture/ports-and-adapters.md) - [Context Map](../contracts/context-map.md) diff --git a/docs/adr/ADR-0007-deterministic-loop-only-operator-stops.md b/docs/adr/ADR-0007-deterministic-loop-only-operator-stops.md index 799675107..c5b05c845 100644 --- a/docs/adr/ADR-0007-deterministic-loop-only-operator-stops.md +++ b/docs/adr/ADR-0007-deterministic-loop-only-operator-stops.md @@ -52,7 +52,7 @@ nor reinstates unlimited operation. Earlier evidence is preserved as history. - An empty claimable queue produces honest **operator-wait** (idle/sanity cycles), never false dormancy. - A genuine regression halts the loop so a human looks — the loop never papers over red. - The guarantee is testable: `tests/scripts/evolve-halt-check.bats` asserts both the gate's behavior and that the skill actually invokes it (no orphaned-primitive regression). -- The same operator-markers-are-authoritative principle generalizes to the **in-session** path: an autonomous-drive directive (a `/goal` Stop-hook, a `/loop`) cannot override a human STOP marker or a pawl HOLD any more than it can self-halt the evolve loop here. See [`pawls.md` § Directive precedence](../contracts/pawls.md#directive-precedence-autonomy-never-overrides-a-human-gate). +- The same operator-markers-are-authoritative principle generalizes to the **in-session** path: an autonomous-drive directive (a `/goal` Stop-hook, a `/loop`) cannot override a human STOP marker or a pawl HOLD any more than it can self-halt the evolve loop here. See `pawls.md` § Directive precedence (historical reference; retired path: `../contracts/pawls.md#directive-precedence-autonomy-never-overrides-a-human-gate`). ## Evidence this is needed diff --git a/docs/adr/ADR-0009-daemon-deletion-in-session-only.md b/docs/adr/ADR-0009-daemon-deletion-in-session-only.md index a19493271..49b942f5d 100644 --- a/docs/adr/ADR-0009-daemon-deletion-in-session-only.md +++ b/docs/adr/ADR-0009-daemon-deletion-in-session-only.md @@ -6,7 +6,7 @@ - **Builds on:** [ADR-0002](ADR-0002-agentops-3-hookless-cdlc-rearchitecture.md) (hookless-first), [ADR-0007](ADR-0007-deterministic-loop-only-operator-stops.md) (deterministic evolve loop) - **Supersedes:** the "Software factory daemon (`ao daemon`)" surface as an AgentOps-shipped capability. -> **Historical-substrate note (added 2026-06-03, ag-xa7q):** This ADR predates the Gas City prune (ag-124p, #679) and names **Gas City** as the reference out-of-session substrate throughout. The *decision* (delete the daemon; adopt a substrate; AgentOps has no sovereign core to protect) stands unchanged — but the substrate has since been renarrated to **NTM + MCP (`ao mcp serve`) + managed-agents (`ao agent`)**; `runtime=gc`, `city.toml`, and `packs/agentops` were removed. Read "Gas City" below as "the out-of-session substrate." Canonical current statement: [docs/3.0.md](../3.0.md). The decision record is preserved as-is for provenance. +> **Historical-substrate note (added 2026-06-03, ag-xa7q):** This ADR predates the Gas City prune (ag-124p, #679) and names **Gas City** as the reference out-of-session substrate throughout. The *decision* (delete the daemon; adopt a substrate; AgentOps has no sovereign core to protect) stands unchanged — but the substrate has since been renarrated to **NTM + MCP (`ao mcp serve`) + managed-agents (`ao agent`)**; `runtime=gc`, `city.toml`, and `packs/agentops` were removed. Read "Gas City" below as "the out-of-session substrate." Statement at the time: docs/3.0.md (historical reference; retired path: `../3.0.md`). The decision record is preserved as-is for provenance. ## Active disposition — 2026-09-06 CDLC adoption @@ -89,8 +89,8 @@ This ADR is accepted when: ## References -- [AgentOps 3.0: the north star](../3.0.md) -- [The Canonical Loop Model](../architecture/canonical-loop-model.md) +- AgentOps 3.0: the north star (historical reference; retired path: `../3.0.md`) +- The Canonical Loop Model (historical reference; retired path: `../architecture/canonical-loop-model.md`) - [ADR-0002: Hookless-First CDLC Rearchitecture](ADR-0002-agentops-3-hookless-cdlc-rearchitecture.md) - [`.agents/discovery/2026-05-24-gvkj6-e2e-proof.md`](https://github.com/boshu2/agentops/blob/main/.agents/discovery/2026-05-24-gvkj6-e2e-proof.md) — the end-to-end proof (bead `soc-5jwah`) - Beads: `soc-j7a5q` (this ADR), `soc-2rtm0` (the orchestration-CLI rip), `soc-5jwah` (the e2e proof + upstream-GC gap) diff --git a/docs/adr/ADR-0017-loop-as-control-flow-not-knowledge.md b/docs/adr/ADR-0017-loop-as-control-flow-not-knowledge.md index 44254c3c3..4078095c3 100644 --- a/docs/adr/ADR-0017-loop-as-control-flow-not-knowledge.md +++ b/docs/adr/ADR-0017-loop-as-control-flow-not-knowledge.md @@ -1,11 +1,34 @@ # ADR-0017: The Loop Is Control Flow, Not Knowledge -- **Status:** Accepted, amended for review selection 2026-09-08 (2026-09-03) +- **Status:** Accepted, amended for native execution 2026-09-10 (2026-09-03) - **Author:** AgentOps maintainers - **Builds on:** [ADR-0004](ADR-0004-corpus-moat-unproven-position-on-the-system.md) (corpus moat unproven, position on the verification system), [ADR-0011](ADR-0011-escape-corpus-compounding-unproven-structural-starvation.md) (escape-corpus compounding demoted to hypothesis) - **Origin:** `docs/plans/2026-09-03-loop-restore.md` (this decision's intent source), and the 2026-09-02 Train 1 run, where the repair loop had to be improvised by hand -## Active lean-harness amendment — 2026-09-09 +## Active native-default amendment — 2026-09-10 + +The caller selected native agent execution with zero mandatory AgentOps skills. +This supersedes RPI as the default entrypoint, not the exact-content judgment +bar. A native goal carries continuity, BD carries work and handoffs, and the +model chooses useful actions within accepted scope. AO provides deterministic +tools on demand. A fresh author-distinct reviewer judges the final change; +invoking the Validate skill is optional. + +Keep the existing skill library and explicit full installations. Selective +linking makes individual guidance available without requiring a bundle. RPI +remains an explicitly selected workflow; its dependency graph is unchanged. +No new scheduler, aggregate controller, knowledge platform or lifecycle command +is authorized. Native limits must be real runtime controls; goal prose cannot +claim enforcement. Historical missing-proof outcomes remain unwaived. + +This amendment responds to observed process overhead and the caller's decision +to test direct execution. A native coding demonstration establishes usability, +not comparative skill uplift. Evaluation must distinguish endpoint checks from +independently accepted work, retain failures and missing proof, and make +association and evidence-verification limits visible. Subsequent comparative +evidence may justify selectively adding or removing guidance. + +## Lean-harness amendment — 2026-09-09 The caller approved a short RPI charter with on-demand Plan, Implement, Validate and Memory. This supersedes the once-only phase lock, mandatory diff --git a/docs/agent-workflow-reference.md b/docs/agent-workflow-reference.md index 9c93bf00b..8428f0efc 100644 --- a/docs/agent-workflow-reference.md +++ b/docs/agent-workflow-reference.md @@ -1,17 +1,17 @@ # Agent workflow reference -[AGENTS.md](../AGENTS.md) and the [RPI charter](../skills/rpi/SKILL.md) direct the -agent to own the authorized outcome through finish. Use the native agent and +[AGENTS.md](../AGENTS.md) directs the agent to own the authorized outcome through +finish. Zero AgentOps skills are required. Use the native agent and shell; the tracker, Git and runtime keep their existing authority. -1. Use the existing intent. Invoke Plan only for missing intent or consequential +1. Use the existing intent. Plan only for missing intent or consequential uncertainty. A trivial change needs no Plan, Recall or Learn worksheet. 2. Implement the smallest useful change, repair known defects directly, and revise an approach when evidence disproves its assumption within unchanged outcome and scope. Changing acceptance needs caller authority. 3. Run cheap discriminating checks while editing, then required integration checks. Reserve capacity for finishing, final judgment and needed repairs. -4. Obtain fresh author-distinct final Validate over the exact subject. Same-family +4. Obtain fresh author-distinct final judgment over the exact subject. Same-family is default; cross-model is opt-in, with no fixed ten-minute cap. PASS requires all acceptance and empty `not_checked`; evidence gaps remain visible. 5. Repair known findings within real bounds and revalidate the changed subject. @@ -26,6 +26,9 @@ work establishes benefit. This lean route uses public or already-cleared inputs and claims no native restricted-source enforcement. Specialists, anti-ceremony audits, factories and outer-goal guidance are optional. +The [RPI charter](../skills/rpi/SKILL.md) is an explicitly selected workflow; +native goals and direct coding do not require it. A skill earns its context +cost by resolving a task-specific need, not by occupying a phase in a sequence. No new scheduler, command or process ledger is needed. The grandfathered pure fixed-dispatch adapter is separately described in its own reference. diff --git a/docs/architecture/go-cli.md b/docs/architecture/go-cli.md index 6e559f4a7..cfa91fbd4 100644 --- a/docs/architecture/go-cli.md +++ b/docs/architecture/go-cli.md @@ -1,7 +1,8 @@ # Go CLI architecture The `ao` binary supplies deterministic repository checks, evidence inspection, -and operational utilities. Semantic judgment belongs to the Validate skill; +and operational utilities. Semantic judgment belongs to a fresh native reviewer; +the Validate skill is optional guidance for that review. Git, delivery, retries, work ownership, and continuation belong to the caller. This page maps an active command from CLI input to effect and result. @@ -86,11 +87,12 @@ never a prebuilt `cli/bin/ao`. A gate PASS is a deterministic fact, not a semantic verdict. -## Evidence contracts and the Validate boundary +## Evidence contracts and the judgment boundary - `schemas/verdict.v2.schema.json` and `schemas/subject-manifest.v1.schema.json` declare the evidence contracts (see [docs/SCHEMAS.md](../SCHEMAS.md)). -- The Validate skill is the semantic author of `verdict.v2`. The +- A fresh reviewer is the semantic author of `verdict.v2`, whether reviewing + directly or using the optional Validate skill. The `ao provenance` evidence leaves compute subject identity and structurally verify and atomically store a supplied judgment through `internal/evidence`, reusing `internal/verdictcheck` canonical/strict primitives. Evidence helper @@ -120,8 +122,9 @@ A gate PASS is a deterministic fact, not a semantic verdict. The in-repo measurement surface (the former eval command family) was retired unconsumed — no gate, workflow, or script ran it (see [docs/MIGRATION.md](../MIGRATION.md)). Learn remains an optional, off-path -consumer of durable verdicts: the `learn` skill reads verdict evidence and -writes TTL'd scratch notes, never a source of record. Rubric projections are +consumer of authorized episodes and other evidence: the `learn` skill can +update reviewed external topic pages under the Memory contract. No blind TTL +or per-session lesson is required. Rubric projections are still leak-guarded by `schemas/outcomes-rubric.v1.schema.json` (`internal/evalsubstrate`), and true holdout rubrics live in the external measurement register, not this repo. Measurement, when a repository wants it, diff --git a/docs/architecture/rpi-traversal.md b/docs/architecture/rpi-traversal.md index 394c3e66b..edc6bab7b 100644 --- a/docs/architecture/rpi-traversal.md +++ b/docs/architecture/rpi-traversal.md @@ -1,6 +1,10 @@ # RPI traversal -This page owns exact evidence semantics for the lean [RPI operating charter](../../skills/rpi/SKILL.md). +This page owns exact evidence semantics for native execution and the optional +[RPI operating charter](../../skills/rpi/SKILL.md). No AgentOps skill invocation +is required for native coding or a fresh independent review. The skill names +below describe the selected RPI workflow; the same evidence bar applies to +direct native work. AgentOps is the operations layer for agentic engineering: portable skills and independent judgment, not a scheduler, tracker, Git workflow or autonomous controller. [Vocabulary](../contracts/ubiquitous-language.md) names the source @@ -66,7 +70,7 @@ verify-manifest, snapshot-intent, digest, store-verdict, verify-verdict and verify-subject helpers establish structural facts, not semantic judgments. They add no tracker, queue, Git delivery or runtime authority. -## Fresh final Validate +## Fresh final judgment (optional Validate guidance) A fresh author-distinct context reads the exact subject, unchanged acceptance and authorized evidence independently. A new role in the author's context is @@ -82,7 +86,7 @@ NOT_PROVEN. Risk deepens inspection rather than automatically adding families or specialists. Use [model-dispatch](../../skills/agent-native/references/model-dispatch.md) with caller/native bounds, no fixed ten-minute cap and no renewed allowance. -Validate derives subject identity at the beginning and end, confirms intent +The fresh reviewer derives subject identity at the beginning and end, confirms intent continuity and complete changed-path coverage, and judges each acceptance criterion against real evidence. Proven out-of-scope change or failed acceptance is FAIL. Subject mutation, digest mismatch, incomplete coverage, missing @@ -102,11 +106,11 @@ intent; weakening the oracle to get green is FAIL. Keep all necessary findings from selected judges with stable IDs/classes. Recurrence or unknown cause calls for causal examination; counts alone cannot prove progress, regression or a wrong design. Preserve disagreement; neither -majority vote nor the author's preferred review certifies PASS. Validate reads +majority vote nor the author's preferred review certifies PASS. The reviewer reads and returns judgment; implementers repair and obtain new fresh judgment over changed exact content. -When requested by a caller or declared consumer, Validate authors `verdict.v2` +When requested by a caller or declared consumer, the reviewer authors `verdict.v2` and uses `ao provenance store-verdict` for structural verification and atomic storage in the explicit protected external non-Git root. No default write to consumer `.agents/` is implied. Legacy requested proof remains preserved. diff --git a/docs/doctrine/effective-feedback-compute.md b/docs/doctrine/effective-feedback-compute.md index acbe11892..2217e0c10 100644 --- a/docs/doctrine/effective-feedback-compute.md +++ b/docs/doctrine/effective-feedback-compute.md @@ -7,7 +7,7 @@ > evidence, not current architecture; the current semantic contract lives in > [rpi-traversal.md](../architecture/rpi-traversal.md). -> **Doctrine.** AgentOps is an agent harness. This doc operationalizes the EFC scaling law (Zhang et al. 2026, *arXiv:2605.29682*) into the rules the harness runs by. Source mining: [`.agents/research/2026-06-16-efc-agent-harness-scaling-laws.md`](https://github.com/boshu2/agentops/blob/main/.agents/research/2026-06-16-efc-agent-harness-scaling-laws.md) (local corpus). Evidence tier: [`the-science.md`](../the-science.md) Part 10. +> **Doctrine.** AgentOps is an agent harness. This doc operationalizes the EFC scaling law (Zhang et al. 2026, *arXiv:2605.29682*) into the rules the harness runs by. Source mining: [`.agents/research/2026-06-16-efc-agent-harness-scaling-laws.md`](https://github.com/boshu2/agentops/blob/main/.agents/research/2026-06-16-efc-agent-harness-scaling-laws.md) (local corpus). Evidence tier: `the-science.md` (historical reference; retired path: `../the-science.md`) Part 10. > > **The claim in one line:** harness success scales with *useful feedback*, not *spend*. Same tokens, better feedback steering → ~3× success (0.27 → 0.90 at matched budget). Optimize the feedback, not the budget. > @@ -91,6 +91,6 @@ Same claim, two altitudes: **compounding requires useful feedback, not volume ## See also -- [`the-science.md`](../the-science.md) Part 10 — EFC as Tier-A external evidence. +- `the-science.md` (historical reference; retired path: `../the-science.md`) Part 10 — EFC as Tier-A external evidence. - [`architecture/rpi-traversal.md`](../architecture/rpi-traversal.md) — the current one-traversal contract (the seven-move loop this page cites is retired). - [`knowledge-flywheel.md`](../knowledge-flywheel.md) — the M factor at corpus scale. diff --git a/docs/domain-practice-packets.md b/docs/domain-practice-packets.md index ad3e921b2..be28f7a43 100644 --- a/docs/domain-practice-packets.md +++ b/docs/domain-practice-packets.md @@ -112,7 +112,7 @@ Before a packet is used in a demo, PMF scenario, or release claim, verify: ## Example -See [AgentOps 3.0 Domain/Practice Packet](examples/agentops-3-domain-practice-packet.md). +Historical example: AgentOps 3.0 Domain/Practice Packet (retired path: `examples/agentops-3-domain-practice-packet.md`). The matching launch storyboard is [AgentOps 3.0 Council Demo Storyboard](examples/agentops-3-council-demo-storyboard.md). The sample verdict artifact is diff --git a/docs/first-value-path.md b/docs/first-value-path.md index 25dcb4bd2..c8a9202c9 100644 --- a/docs/first-value-path.md +++ b/docs/first-value-path.md @@ -1,18 +1,21 @@ # First value path -1. Give RPI one concrete behavior. -2. Plan writes normal and edge Given/When/Then examples, non-goals, evidence, - and an explicit write scope. -3. Implement runs one bounded experiment and records actual changed paths and - factual checks. -4. A fresh context runs Validate against the exact subject manifest. -5. Validate returns the result; RPI reports it and stops. Persist `verdict.v2` - only when machine-readable evidence was requested. +1. Give your native coding agent a concrete behavior and acceptance examples. +2. Let it inspect the relevant code, implement the change and run the repository's + checks. Planning and specialist guidance are available when they help. +3. Have a fresh, author-distinct context review the exact change against the + unchanged acceptance and evidence. +4. Repair concrete findings and review the changed content. Finish when the + accepted behavior is proven; report missing evidence honestly. + +No AgentOps skill installation or RPI invocation is required. `ao quick-start` +explains this path and `ao demo` prints a sample task. `ao` supplies deterministic +checks and evidence utilities on demand; the coding agent supplies judgment. +Use a native goal for continuity and your existing tracker for work and handoffs. For a small task this should be one reviewable change and one independent -judgment—not a project-management ceremony. Optional premortems, councils, -specialists, trackers, and factory adapters are chosen by the caller only when -they add value. +judgment. Persist `verdict.v2` only when a caller or declared consumer needs +machine-readable evidence, using protected external non-Git storage. Success is fresh independent judgment bound to acceptance and content -identities. Pushing or releasing that content is a separate repository decision. +identities. Pushing or releasing that content follows repository policy. diff --git a/docs/how-it-works.md b/docs/how-it-works.md index 3c1b2668e..e6eb63c63 100644 --- a/docs/how-it-works.md +++ b/docs/how-it-works.md @@ -1,26 +1,37 @@ # How it works -RPI is a thin coordinator with one bounded repair loop (ADR-0017). +Your native coding agent owns the authorized change. AgentOps supplies a short +repository brief, deterministic AO tools and optional specialist guidance. +No RPI invocation or skill installation is required. ```text -Plan once -> Implement once -> fresh Validate -> repair under the convergence law -> Report +Accepted intent -> native implementation and checks -> fresh independent judgment -> finish ``` -Plan turns intent into testable acceptance and a bounded write scope. Implement -runs one experiment and packages the observed result. Validate computes exact -content identity, checks that changed-path coverage is complete, checks scope and -acceptance, and asks one distinct fresh context for criterion-level judgment. +Keep behavior and scope in the existing issue or conversation. Use normal and +edge acceptance examples when useful. Implement, check and directly repair +known defects. A fresh author-distinct reviewer reads the exact change and +judges every acceptance criterion against evidence. Passing tests alone cannot +prove accepted work. -Validate is the sole verdict writer. It writes canonical JSON to a temporary -file in the destination directory, flushes it, and atomically renames it to: +PASS requires complete checked scope, distinct identities, attested freshness +and evidence for acceptance. Failed acceptance is FAIL; missing proof remains +NOT_PROVEN. A repair changes the subject and needs a new judgment of that content. + +When a caller or declared consumer needs durable evidence, the reviewer authors +`verdict.v2` and AO structurally verifies and atomically stores it in an explicit +protected external non-Git root: ```text -.agents/ao/verdicts/sha256/.json +/verdicts/sha256/.json ``` -An identical existing artifact is success. Different content under the same -digest is an integrity failure and produces `NOT_PROVEN`. +Identical stored content is idempotent. A digest collision with different +content fails integrity checks. Storage and check success cannot substitute +for a semantic judgment. Legacy `.agents/` evidence remains preserved under +owner policy. -Optional strategies may improve inputs. Optional factory adapters may dispatch -explicit disjoint packets once. Neither can add a core phase, alter a verdict, -or choose continuation. +The [RPI workflow](../skills/rpi/SKILL.md) remains available by explicit choice. +Native goals supply continuity and the existing tracker stores work and +handoffs. Memory is optional retrieval or separately scoped curation. Skills, +reviews and learning earn their cost through later engineering outcomes. diff --git a/docs/index.md b/docs/index.md index 908bc0a83..93b51e3ee 100644 --- a/docs/index.md +++ b/docs/index.md @@ -7,7 +7,7 @@ that own work, execution, or delivery. It turns one intent into one evidence-bound engineering judgment: ```text -RPI -> Plan -> Implement -> fresh Validate -> repair to convergence -> report +Accepted intent -> native implementation and checks -> fresh independent judgment -> finish ``` The product supplies behavior-first planning, one bounded implementation @@ -15,6 +15,9 @@ experiment, deterministic content identity, fresh independent judgment, and a standalone content-addressed verdict. It does not own retries, work queues, trackers, Git, CI, release, or delivery. +Native execution requires zero AgentOps skills. Start with your coding agent +and repository checks; load specialist guidance for a concrete need. + ## Start here - [First value path](first-value-path.md) diff --git a/docs/install-day2-ops.md b/docs/install-day2-ops.md index fed15690f..f2c9079d9 100644 --- a/docs/install-day2-ops.md +++ b/docs/install-day2-ops.md @@ -1,6 +1,19 @@ # Install And Day-2 Operations -AgentOps 3.3 supports three install paths: +The default is your native coding agent plus the `ao` binary, with zero +mandatory AgentOps skills. Install AO and read the native quickstart: + +```bash +go install github.com/boshu2/agentops/cli/cmd/ao@latest +ao quick-start +ao demo +``` + +These guidance commands do not write skills, hooks, or project state. `ao init` +is optional local evidence setup, not a prerequisite. Development features can +be built from this checkout with `cd cli && go install ./cmd/ao`. + +Three optional skill installation paths remain supported: - `npx skills@latest add boshu2/agentops --all -g` — universal; one command installs the skills into all your coding agents. @@ -35,9 +48,8 @@ The plugin and `npx skills@latest add boshu2/agentops --all -g` install the gene ## Maintainer / contributor: the `ao` binary -The `ao` CLI is optional: `fitness` and `using-gc` call it directly -(see the table above); the rest of the skills work without it. Install it on -its own: +Install `ao` on its own; no skill linking is needed for native execution. +Selected skills may also require it as listed above: ```bash brew tap boshu2/agentops https://github.com/boshu2/homebrew-agentops @@ -51,14 +63,15 @@ To track skills from a local checkout instead of a release bundle, run ## Install (source checkout) -Install the optional `ao` CLI, clone AgentOps, and link its skills: +To make selected guidance discoverable, install the CLI, clone AgentOps, and +link those skills: ```bash brew tap boshu2/agentops https://github.com/boshu2/homebrew-agentops brew install agentops git clone https://github.com/boshu2/agentops.git ~/.local/share/agentops cd ~/.local/share/agentops -ao skills link +ao skills link --skill test --skill refactor ``` Without Homebrew: @@ -68,10 +81,15 @@ git clone https://github.com/boshu2/agentops.git ~/.local/share/agentops cd ~/.local/share/agentops/cli go install ./cmd/ao cd .. -"$(go env GOPATH)/bin/ao" skills link +"$(go env GOPATH)/bin/ao" skills link --skill test --skill refactor ``` -The command links each canonical `skills//` directory into +Repeat `--skill` to select exact catalog names. Unknown or invalid names fail +before any links are created. Omit selectors for the existing full-library +behavior; choosing a subset does not remove previously installed skills. +Use `--dry-run` to preview and `--dest /path/to/skills` for one discovery root. + +The command links selected canonical `skills//` directories into `~/.agents/skills` and every detected runtime skills root. It refuses to replace real directories, foreign links, or user-owned skills. @@ -80,11 +98,12 @@ real directories, foreign links, or user-owned skills. ```bash cd ~/.local/share/agentops git pull --ff-only -ao skills link +ao skills link --skill test --skill refactor ``` Existing links immediately see edits to their targets. Rerunning `ao skills -link` adds newly introduced skills and reports conflicts. It does not copy the +link` with the same selection restores selected links and reports conflicts; +without selectors it also adds newly introduced skills. It does not copy the corpus or refresh a plugin cache. ## Audit diff --git a/docs/profiles/README.md b/docs/profiles/README.md index b59c78f8d..769b2198c 100644 --- a/docs/profiles/README.md +++ b/docs/profiles/README.md @@ -10,9 +10,9 @@ Profiles are **documentation groupings**, not executable configs. They organize AgentOps skills by domain so you can quickly find which skills are relevant to your work. -For AgentOps 3.0 workflow recipes such as `product-council`, -`engineering-team`, and `always-on-substrate`, see -[Activation Profiles](../activation-profiles.md). Activation profiles are +AgentOps 3.0 workflow recipes such as `product-council`, +`engineering-team`, and `always-on-substrate` were documented in +Activation Profiles (historical reference; retired path: `../activation-profiles.md`). Activation profiles are product recipes with explicit inputs, commands, and expected artifacts; this page remains the role taxonomy. diff --git a/docs/releases/2026-06-08-v3.1.0-notes.md b/docs/releases/2026-06-08-v3.1.0-notes.md index 0d66429e4..af49a29f8 100644 --- a/docs/releases/2026-06-08-v3.1.0-notes.md +++ b/docs/releases/2026-06-08-v3.1.0-notes.md @@ -2,7 +2,7 @@ ## Highlights -AgentOps 3.1 is the **packaging-and-gate-discipline** minor on top of the hookless 3.0 core: the release pipeline itself became the product. Three things hardened together — the **release gate moved native** (the inline bash checks were ported to a single Go gate that the pre-push hook runs as the release authority, with workflow-parity enforcement so a check can't pass locally while drifting in CI), **close-admission tightened** (the verdict gate now counts distinct validator *families* with author exclusion, so a self-graded "looks good" no longer admits a close), and the **skill corpus was canonicalized** (ghost skills materialized into the tree, registries and counts regenerated from one source, and bundled image copies forced back to byte identity). On top of that discipline, the three image recipes (Claude, Codex, and Gemini/AGY) ship as real, smoke-proven install paths. See [docs/3.1.md](../3.1.md) for the full narrative. +AgentOps 3.1 is the **packaging-and-gate-discipline** minor on top of the hookless 3.0 core: the release pipeline itself became the product. Three things hardened together — the **release gate moved native** (the inline bash checks were ported to a single Go gate that the pre-push hook runs as the release authority, with workflow-parity enforcement so a check can't pass locally while drifting in CI), **close-admission tightened** (the verdict gate now counts distinct validator *families* with author exclusion, so a self-graded "looks good" no longer admits a close), and the **skill corpus was canonicalized** (ghost skills materialized into the tree, registries and counts regenerated from one source, and bundled image copies forced back to byte identity). On top of that discipline, the three image recipes (Claude, Codex, and Gemini/AGY) ship as real, smoke-proven install paths. See docs/3.1.md (historical reference; retired path: `../3.1.md`) for the full narrative. ## Upgrade Notes diff --git a/docs/releases/2026-07-03-v3.2.0-notes.md b/docs/releases/2026-07-03-v3.2.0-notes.md index 06e5c0225..3e3499c4d 100644 --- a/docs/releases/2026-07-03-v3.2.0-notes.md +++ b/docs/releases/2026-07-03-v3.2.0-notes.md @@ -2,7 +2,7 @@ ## Highlights -AgentOps 3.2 makes verification the front door of the product. `ao verify` reviews a change with a model from a different family than the one that wrote it, binds the verdict to the provenance ledger, and can install that same pre-push ratchet in any repo; `ao done` closes work only against such a verdict. The reviewer now runs as a standing multi-model service that reports its true state: failover engages only during an outage and labels its verdicts degraded, a byte-identical rebase reuses its existing verdict (REBOUND) instead of paying for a fresh review, and the opt-in strict quorum reports UNAVAILABLE when a second qualified reviewer family is missing. Reviews also leave a record — `ao membrane catch/recall/triage` stores what reviewers caught, recalls it by domain, and reports whether the same class keeps recurring. The default CLI build slims to the verification and bookkeeping core: `ao rpi` and the v1 memory commands are removed, the bd/Dolt tracker is retired in favor of `br` + `bv`, and the skill catalog drops from 105 to 63. [docs/3.2.md](../3.2.md) has the full narrative. +AgentOps 3.2 makes verification the front door of the product. `ao verify` reviews a change with a model from a different family than the one that wrote it, binds the verdict to the provenance ledger, and can install that same pre-push ratchet in any repo; `ao done` closes work only against such a verdict. The reviewer now runs as a standing multi-model service that reports its true state: failover engages only during an outage and labels its verdicts degraded, a byte-identical rebase reuses its existing verdict (REBOUND) instead of paying for a fresh review, and the opt-in strict quorum reports UNAVAILABLE when a second qualified reviewer family is missing. Reviews also leave a record — `ao membrane catch/recall/triage` stores what reviewers caught, recalls it by domain, and reports whether the same class keeps recurring. The default CLI build slims to the verification and bookkeeping core: `ao rpi` and the v1 memory commands are removed, the bd/Dolt tracker is retired in favor of `br` + `bv`, and the skill catalog drops from 105 to 63. docs/3.2.md (historical reference; retired path: `../3.2.md`) has the full narrative. ## Upgrade Notes @@ -105,7 +105,7 @@ AgentOps 3.2 makes verification the front door of the product. `ao verify` revie ### Docs and Onboarding -- Added: [docs/3.2.md](../3.2.md), the release narrative, plus a quick-start that lands a first verdict in one command. +- Added: docs/3.2.md (historical reference; retired path: `../3.2.md`), the release narrative, plus a quick-start that lands a first verdict in one command. - Added: a loop-map doc indexing the five loop framings, and §6 conformance-pattern documentation for workflow authors. - Changed: the root agent docs were reconciled to executable truth, with source-of-truth precedence stated inline and stale tracker and hook-era wording swept. - Changed: README claims now cite evidence docs, and unproven-claim language is mechanically banned from live docs by a gate. diff --git a/docs/runbooks/outcomes-grading.md b/docs/runbooks/outcomes-grading.md index f9b61aeff..c65e16a87 100644 --- a/docs/runbooks/outcomes-grading.md +++ b/docs/runbooks/outcomes-grading.md @@ -115,6 +115,6 @@ once. A non-positive budget = no ceiling (no-op). **Dev-split scores never burn* ## See also -- [Outcomes Rubric Projection Contract](../contracts/outcomes-rubric-projection.md) -- [Eval Verdict Pipeline Contract](../contracts/eval-verdict-pipeline.md) +- Outcomes Rubric Projection Contract (historical reference; retired path: `../contracts/outcomes-rubric-projection.md`) +- Eval Verdict Pipeline Contract (historical reference; retired path: `../contracts/eval-verdict-pipeline.md`) - `~/.agents/evals/SCHEMA.md` — the LOCKED substrate this lane projects from (never relitigated) diff --git a/evals/skills-rpi/README.md b/evals/skills-rpi/README.md index e99878956..2fc18e4a3 100644 --- a/evals/skills-rpi/README.md +++ b/evals/skills-rpi/README.md @@ -104,10 +104,20 @@ Do not add native and Harbor usage together or sum repeated cumulative copies. For an ordinary coding observation, the same reader accepts explicit `--session /absolute/native/session.jsonl` inputs. This collects execution facts; -it does not automatically discover or join arbitrary sessions to Git acceptance -or reviews. Existing caller-owned check/judgment evidence must supply that -relationship. Historical imports remain observational, with missing baseline -fields visible. They do not establish skill efficiency. +an optional one-subject join accepts the existing provenance flags (`--root`, +`--manifest`, `--intent`, `--author-context-id`, `--required-profiles`, +`--allowed-provider`, `--evidence-root`, and repeated `--verdict`). It reuses +`evidence.VerifyJudgments` over caller-owned exact subject, acceptance and native +reviewer receipts. The join also requires repeatable `--required-criterion` +inputs for the caller's complete acceptance ID set; partially omitted, duplicate +or unexpected verdict criteria remain unproven. It then requires a unique observed author association and +completed execution before reporting accepted work. No skill invocation is +required. Endpoint rewards, verified FAIL, missing proof and unfinished work +remain separate. See [the readout contract](readout.md) for the complete example, +denominators and limitations, including unresolved arbitrary criterion evidence +references. It does not automatically discover or join arbitrary sessions to +Git acceptance or reviews. Historical imports remain observational, with missing +baseline fields visible. They do not establish skill efficiency. A small baseline pilot normally ends with **insufficient evidence** about uplift. Keep case-level outcomes, uncertainty, all unsuccessful costs and missing diff --git a/evals/skills-rpi/readout.md b/evals/skills-rpi/readout.md index 774c9f209..0feed6084 100644 --- a/evals/skills-rpi/readout.md +++ b/evals/skills-rpi/readout.md @@ -98,12 +98,105 @@ One small development batch is not a powered or held-out skill-benefit claim. The automated pilot recommendation is `insufficient-evidence`; the specialist can make a narrower supported, explicitly provisional maintenance decision. -Native verifier reward 1 is an endpoint success. The reader leaves independent -completion, worker false completion and feasibility unknown. A required fresh +Native verifier reward 1 is an endpoint success. Worker false completion and +feasibility remain unknown. A required fresh handoff or exact-subject judgment is not waived by a green executable oracle. Native rewards are retained for inspection; the reader never parses worker prose into acceptance. +For an ordinary native coding observation, optionally join one explicitly +selected subject using the existing provenance judgment inputs. No skill +invocation or new trial is required. The acceptance, subject, author, complete +required acceptance ID set and reviewer profiles must come from the caller, +independently of candidate verdicts: + +```bash +# From cli/. Add --job for a captured Harbor trial when applicable. +go run ./cmd/skill-trial-report \ + --session /protected/native/author.jsonl \ + --root /absolute/candidate \ + --manifest /protected/proof/manifest.json \ + --intent /protected/proof/acceptance.intent \ + --author-context-id native-author-id \ + --required-profiles /protected/proof/profiles.json \ + --required-criterion value --required-criterion scope \ + --allowed-provider openai \ + --evidence-root /protected/proof \ + --verdict /protected/proof/CONTENT_DIGEST.json > /protected/native-report.json +``` + +Render from the repository root: + +```bash +python evals/skills-rpi/readout.py --report /protected/native-report.json +``` + +The profile file uses the existing `ao provenance verify-judgments` format: an object with a `profiles` +array of `id`, `runtime`, `model`, `family`, and `effort` strings. Use an empty +effort only when no actual-effort requirement was selected. Repeat `--verdict` +for required legs and `--allowed-provider` for independently authorized +providers; these flags launch nothing. Supply `--base-manifest` for deletions. +Missing required verdicts produce an unproven report. Partial judgment options +are usage errors; invalid or missing subject/receipt/transcript evidence remains +visible in the report's `work.problems` or original `work.judgments` result. + +Repeat `--required-criterion` for **every acceptance criterion ID**, as supplied +by the caller before inspecting the candidate verdict. The native work join +requires a nonempty, unique set of unpadded IDs. Every original verdict must +cover that exact set once: a PASS listing `value` while omitting required +`scope` remains unproven, as do duplicate IDs or unexpected IDs. The result +retains the caller's `required_criteria` and every original leg with its coverage +problems. Missing the flag is a usage error; direct report API calls without a +selected criterion policy can never report accepted work. + +The caller owns completeness of that ID set; the fresh reviewer still maps +each ID to the immutable acceptance and actual supporting evidence. This is +exact ID coverage, not automatic extraction or interpretation of intent prose. +The existing provenance API keeps its prior behavior when no optional criterion +policy is selected; that legacy coverage alone is insufficient for native work +acceptance. + +The native Go reader calls `evidence.VerifyJudgments`, the existing provenance +owner. It verifies the expected subject and acceptance, original verdict +integrity, distinct author/reviewer identities, freshness, required profiles, +hash-bound judgment receipts, exact required criterion coverage and native +reviewer completion. Verdicts, receipts +and reviewer transcripts retain that owner's private non-Git root and size +restrictions. It **does not mechanically resolve arbitrary criterion evidence +references**. The fresh reviewer owns their semantic assessment. This reader +reports supplied independent judgments and never issues a semantic verdict. + +`work.status` is `accepted` only with completed native author execution and +satisfied independent PASS coverage of the caller's complete acceptance ID set. +A correctly bound, fully covered independent FAIL remains +`failed` even though PASS coverage is unsatisfied. Other incomplete or invalid +proof is `not_proven`; original legs and problems remain visible in every case. +`work.execution` separately reports `completed`, `failed`, `unfinished` or +`unknown`. A timed-out attempt cannot become accepted because its code or later +review passes. Missing native author completion is unknown; terminal markers +whose ordering the accounting reader cannot establish also remain unknown. +The recognized diagnostic for decreasing cumulative usage counters remains in +native accounting and does not invalidate known author identity or execution; +aggregate usage and billing remain unproven. Parse/identity ambiguity and +unsupported abort/error ordering still block acceptance. + +Association requires the explicitly supplied native author identity. One +session found in one trial joins that trial. Different copies of a session or +an identity shared by several trials are ambiguous; no favorable retry is +selected. A standalone session stays visible under `native_work` without an +invented trial, arm, elapsed interval or cost. Unmatched trials retain unproven +acceptance regardless of their endpoint reward. This association is a declared +caller-selected relationship, not automatic proof of which session wrote Git +content. Reported source hashes and exact subject/acceptance identify its bounds. + +Per-arm `independent_work` shows `known_accepted`, `known_failed`, observed +`not_proven`, `unobserved_assignments`, the all-assigned denominator and the +known accepted fraction of it. A zero known count is not proof of zero actual +accepted work when coverage is missing. `independently_completed_outcomes` +remains null until every assignment has a known accepted/failed disposition. +The Python view consumes the native Go report; it does not reverify verdicts or +promote comparison receipts or scalar rewards into independent acceptance. + For validation fixtures, E2 captures the separate verifier's `verifier/grade.json` in `trial.documents` with `path`, `sha256` and parsed `data`. The readout consumes its `case_results` only when the external receipt, native separate-verifier @@ -131,7 +224,7 @@ do not rehabilitate an invalid paired comparison or establish general uplift. The same source's `not_checked` entries join `remaining_workflow_gaps` with their job and source reference. Missing gap coverage remains unknown. Case-level truth does not prove a fresh native handoff or independent completion of the workflow; -`independently_completed_outcomes` remains unknown. +only the explicit native work join above supplies that separate evidence. Cost and timing retain their measurement windows: diff --git a/evals/skills-rpi/readout.py b/evals/skills-rpi/readout.py index e5417cc44..65255dab7 100644 --- a/evals/skills-rpi/readout.py +++ b/evals/skills-rpi/readout.py @@ -216,6 +216,25 @@ def validation_cases(rows): return result +def trial_work(work, trial, directory_count): + """Present the Go reader's explicit association, never infer one from reward.""" + if (isinstance(work, dict) and work.get("association") == "trial" + and work.get("trial_directory") == trial.get("directory") and directory_count == 1 + and work.get("author_context_id") in trial.get("session_ids", []) + and work.get("status") in ("accepted", "failed", "not_proven")): + return deepcopy(work) + return {"status": "not_proven", "problems": ["independent work not supplied or not uniquely associated with this trial"]} + + +def work_counts(rows, denominator): + statuses = Counter(row["independent_work"]["status"] for row in rows) + accepted = statuses["accepted"] + return {"known_accepted": accepted, "known_failed": statuses["failed"], "not_proven": statuses["not_proven"], + "unobserved_assignments": max(0, denominator - len(rows)) if denominator is not None else None, + "denominator": denominator, + "known_accepted_rate_all_assignments": accepted / denominator if denominator else None} + + def uncertainty(pairs, *, n_required=None, resamples=10000): if not pairs: return {"status": "unavailable", "reason": "no comparable endpoint pairs"} @@ -268,6 +287,7 @@ def build(report, receipts=None, *, control="control", treatment="treatment", n_ if (copy.get("accounting") or {}).get("diagnostics"): suspect_sessions.add(session.get("id")) seen_jobs = Counter(Path(job["directory"]).name for job in report.get("jobs", [])) + seen_trials = Counter(trial.get("directory") for job in report.get("jobs", []) for trial in job.get("trials", [])) for job in report.get("jobs", []): arm = job.get("arm", "") arms[arm].append(job) @@ -301,6 +321,7 @@ def build(report, receipts=None, *, control="control", treatment="treatment", n_ "native_rewards": (native.get("verifier_result") or {}).get("rewards"), "verifier_mode_source": verifier_mode_source(trial, receipt), "verifier_grade": grade, + "independent_work": trial_work(report.get("work"), trial, seen_trials[trial.get("directory")]), "native_phase_endpoints": {key: native.get(key) for key in ("environment_setup", "agent_setup", "agent_execution", "verifier")}, "session_ids": trial.get("session_ids", []), @@ -371,6 +392,7 @@ def build(report, receipts=None, *, control="control", treatment="treatment", n_ denominator = max(expected_total, len(rows)) if expected_total is not None else None successes = sum(row["outcome"] == "success" for row in rows) costs = distribution([(row["harbor_agent_result"] or {}).get("cost_usd") for row in rows]) + independent = work_counts(rows, denominator) summaries[arm] = {"expected": expected_total, "expected_known_subtotal": sum(n for n in expected if isinstance(n, int) and n >= 0), "observed": len(rows), "started": sum(job.get("counts", {}).get("started", 0) for job in jobs), "unobserved_expected": max(0, expected_total - len(rows)) if expected_total is not None else None, @@ -378,7 +400,8 @@ def build(report, receipts=None, *, control="control", treatment="treatment", n_ "all_assigned_or_observed_denominator": denominator, "endpoint_successes": successes, "endpoint_success_rate_all_assignments": successes / denominator if denominator else None, - "independently_completed_outcomes": None, + "independently_completed_outcomes": independent["known_accepted"] if denominator == len(rows) and not independent["not_proven"] else None, + "independent_work": independent, "validation_cases": validation_cases(rows), "comparison_eligible_attempts": sum(not row["comparison_exclusions"] for row in rows), "harbor_cost_usd": costs, "elapsed_seconds": distribution([row["elapsed_seconds"] for row in rows]), @@ -390,16 +413,17 @@ def build(report, receipts=None, *, control="control", treatment="treatment", n_ **{key: (copy.get("accounting") or {}).get(key) for key in ("parent_id", "usage", "first_timestamp", "last_timestamp", "latest_turn_state", "diagnostics")}} for copy in session.get("copies", [])]} for session in report.get("sessions", [])] - unmeasured = ["feasibility and substantiated blocking", "worker false completion", "independent semantic acceptance", + unmeasured = ["feasibility and substantiated blocking", "worker false completion", + "independent acceptance outside the explicitly joined subject" if report.get("work") else "independent semantic acceptance", "scope/acceptance drift, recovery, stopping and budget overrun", "content delivery and relevant action", "total billing, orchestration and experimental grading cost"] if not any(summary["validation_cases"]["case_results"] for summary in summaries.values()): unmeasured.extend(["validator false acceptance", "needless blocking on clean candidates"]) return {"recommendation": "insufficient-evidence", - "recommendation_reason": "This pilot readout supports a scoped human maintenance decision; it does not establish held-out benefit, semantic acceptance, equivalence, or complete billed cost.", + "recommendation_reason": "This pilot readout supports a scoped human maintenance decision; it does not establish held-out benefit, equivalence, or complete billed cost. Independent work status applies only to the explicitly joined subject.", "arms": summaries, "paired_outcomes": pairs, "pair_dispositions": pair_dispositions, "uncertainty": uncertainty(pairs, n_required=n_required, resamples=resamples), - "attempts": attempts, "native_sessions": sessions, + "attempts": attempts, "native_sessions": sessions, "native_work": deepcopy(report.get("work")), "receipt_diagnostics": receipts.get("diagnostics", []), "unobserved_receipt_jobs": sorted(name for name in by_job if name not in seen_jobs), "unmeasured": unmeasured, @@ -413,6 +437,7 @@ def build(report, receipts=None, *, control="control", treatment="treatment", n_ "Harbor cost values and ratios are incomplete estimates, even when every attempt has a scalar: the adapter may omit nested sessions. They are separate from native usage and do not establish billing. Missing usage or billing is unknown; zero accepted outcomes makes cost per accepted outcome undefined.", "Infrastructure, missing and ambiguous endpoints are excluded from paired inference, retained in overall assignment accounting. Execution errors count as unsuccessful endpoints when identity is intact.", "Validation case metrics use only receipt-valid, separately captured verifier grades, including failed endpoints. Missing cases or grades remain unknown; recorded case judgments do not establish worker false completion or independent workflow completion.", + "Independent work presents the native Go reader's explicit judgment join; this renderer does not issue or revalidate semantic verdicts. Known accepted counts cover only uniquely associated trials. Missing proof and unobserved assignments stay separate, and total accepted outcomes remain unknown until coverage is complete. Standalone native work has no invented trial or billing assignment.", "Numeric repetition IDs pair observations, not provider randomness. No live enforcement or general uplift claim follows from replay fixtures."]} @@ -425,6 +450,29 @@ def show(value): for arm, row in result["arms"].items(): cost = row["harbor_cost_usd"] lines.append(f"| {arm} | {show(row['expected'])} | {row['observed']} | {row['endpoint_successes']} / {show(row['all_assigned_or_observed_denominator'])} | {json.dumps(row['outcomes'], sort_keys=True)} | ${cost['known_sum']:.4f} / {cost['unknown']} |") + lines += ["", "Independent work from explicitly supplied native judgment evidence:", "", + "| Arm | Known accepted | Known failed | Observed without proof | Unobserved assignments | Denominator |", + "|---|---:|---:|---:|---:|---:|"] + for arm, row in result["arms"].items(): + counts = row["independent_work"] + lines.append(f"| {arm} | " + " | ".join(show(counts[key]) for key in + ("known_accepted", "known_failed", "not_proven", "unobserved_assignments", "denominator")) + " |") + work = result.get("native_work") + if work: + lines += ["", f"Selected native work: {work.get('status', 'not_proven')}; execution {work.get('execution', 'unknown')}; association {work.get('association', 'unknown')}.", + "Author: " + str(work.get("author_context_id") or "unknown") + "."] + judgment = work.get("judgments") or {} + if judgment.get("subject_manifest_digest"): + lines.append("Subject manifest: " + judgment["subject_manifest_digest"] + "; acceptance: " + str(judgment.get("acceptance_digest")) + ".") + if judgment.get("required_criteria"): + lines.append("Required acceptance IDs: " + ", ".join(judgment["required_criteria"]) + ".") + for leg in judgment.get("legs", []): + lines.append(f"- {leg.get('id', 'unknown')}: original {leg.get('verdict', 'unknown')}; " + "; ".join(leg.get("problems", []))) + problems = work.get("problems", []) + judgment.get("problems", []) + if problems: + lines.append("Unproven evidence/association: " + "; ".join(problems) + ".") + else: + lines += ["", "Selected native work: not_proven; no independent judgment join supplied."] lines += ["", "Paired task outcomes (control → treatment):"] lines += [f"- {pair['task']} rep {pair['rep']}: {pair['control_outcome']} → {pair['treatment_outcome']}" for pair in result["paired_outcomes"]] or ["- None."] lines += ["", "Uncertainty: " + json.dumps(result["uncertainty"], sort_keys=True), "", diff --git a/evals/skills-rpi/test_readout.py b/evals/skills-rpi/test_readout.py index 8a2800c19..849380404 100644 --- a/evals/skills-rpi/test_readout.py +++ b/evals/skills-rpi/test_readout.py @@ -145,6 +145,70 @@ def test_native_session_copies_and_parent_child_counters_are_not_added(): assert "session identity shared" in str(result["attempts"]) +def test_independent_work_keeps_endpoint_and_missing_proof_separate(): + report, receipts = fixture(("success", "success")) + trial = report["jobs"][1]["trials"][0] + trial["session_ids"] = ["author"] + report["work"] = {"status": "accepted", "execution": "completed", "association": "trial", + "author_context_id": "author", "trial_directory": trial["directory"], "problems": [], + "judgments": {"satisfied": True, "required_criteria": ["value", "scope"], "legs": [{"id": "review", "verdict": "PASS"}]}} + result = readout.build(report, receipts) + assert result["arms"]["control"]["endpoint_successes"] == 1 + assert result["arms"]["treatment"]["endpoint_successes"] == 1 + assert result["arms"]["control"]["independently_completed_outcomes"] is None + assert result["arms"]["control"]["independent_work"]["not_proven"] == 1 + assert result["arms"]["treatment"]["independently_completed_outcomes"] == 1 + assert result["arms"]["treatment"]["independent_work"]["known_accepted"] == 1 + assert result["arms"]["treatment"]["total_billed_cost_per_accepted_outcome"] is None + assert "accepted; execution completed; association trial" in readout.markdown(result) + assert "Required acceptance IDs: value, scope." in readout.markdown(result) + report["jobs"][1]["counts"]["expected"] = 3 + result = readout.build(report, receipts) + assert result["arms"]["treatment"]["independently_completed_outcomes"] is None + assert result["arms"]["treatment"]["independent_work"]["known_accepted"] == 1 + assert result["arms"]["treatment"]["independent_work"]["unobserved_assignments"] == 2 + assert result["arms"]["treatment"]["independent_work"]["known_accepted_rate_all_assignments"] == pytest.approx(1 / 3) + + +@pytest.mark.parametrize("status,execution,verdict", [ + ("failed", "completed", "FAIL"), ("not_proven", "unfinished", "PASS"), + ("not_proven", "completed", "NOT_PROVEN"), +]) +def test_independent_work_preserves_original_judgments_and_execution(status, execution, verdict): + report, receipts = fixture(("success", "success")) + trial = report["jobs"][1]["trials"][0] + trial["session_ids"] = ["author"] + report["work"] = {"status": status, "execution": execution, "association": "trial", + "author_context_id": "author", "trial_directory": trial["directory"], + "judgments": {"satisfied": verdict == "PASS", "legs": [{"verdict": verdict}]}} + result = readout.build(report, receipts) + assert result["native_work"] == report["work"] + assert result["attempts"][1]["independent_work"] == report["work"] + assert result["arms"]["treatment"]["endpoint_successes"] == 1 + assert result["arms"]["treatment"]["independent_work"]["known_accepted"] == 0 + assert result["arms"]["treatment"]["independent_work"]["known_failed"] == int(status == "failed") + + +def test_standalone_work_stays_visible_without_invented_trial_or_cost(): + report = {"jobs": [], "sessions": [], "limits": [], "work": { + "status": "accepted", "execution": "completed", "association": "standalone_session", + "author_context_id": "native", "trial_directory": "", "judgments": {"satisfied": True}}} + result = readout.build(report) + assert result["native_work"] == report["work"] + assert not result["attempts"] and not result["arms"] + assert "accepted; execution completed; association standalone_session" in readout.markdown(result) + + +def test_ambiguous_work_is_not_assigned_to_any_attempt(): + report, receipts = fixture() + report["work"] = {"status": "not_proven", "execution": "unknown", "association": "ambiguous", + "author_context_id": "shared", "trial_directory": "", "problems": ["ambiguous native copies"]} + result = readout.build(report, receipts) + assert result["native_work"] == report["work"] + assert all(row["independent_work"]["status"] == "not_proven" for row in result["attempts"]) + assert all(arm["independent_work"]["known_accepted"] == 0 for arm in result["arms"].values()) + + def test_no_change_and_degenerate_are_not_equivalence(): report, receipts = fixture(("success", "success")) result = readout.build(report, receipts, n_required=1) diff --git a/mkdocs.yml b/mkdocs.yml index 693fcb33e..4a66d7582 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -186,7 +186,7 @@ nav: - RPI Traversal: architecture/rpi-traversal.md - Operating Loop (former name): architecture/operating-loop.md - Go CLI Architecture: architecture/go-cli.md - - Gas City Factory: architecture/gas-city-factory.md + - Gas City Reliability: operations/gas-city-reliability.md - How It Works: how-it-works.md - Architecture Overview: ARCHITECTURE.md - Component Map: architecture/component-map.md diff --git a/tests/docs/mkdocs-strict-allowlist.txt b/tests/docs/mkdocs-strict-allowlist.txt index 01e35dd09..6a12af378 100644 --- a/tests/docs/mkdocs-strict-allowlist.txt +++ b/tests/docs/mkdocs-strict-allowlist.txt @@ -38,3 +38,24 @@ WARNING - Doc file 'contracts/codex-skill-api.md' contains a link '../../AGENTS WARNING - Doc file 'releases/2026-07-17-v3.3.0-notes.md' contains a link '../../CHANGELOG.md', but the target '../CHANGELOG.md' is not found among documentation files. Did you mean '../CHANGELOG.md'? WARNING - Doc file 'skills/cc-hooks.md' contains a link 'hooks/policy-dispatch.sh', but the target 'skills/hooks/policy-dispatch.sh' is not found among documentation files. WARNING - Doc file 'skills/cc-hooks.md' contains a link 'policies/policies.json', but the target 'skills/policies/policies.json' is not found among documentation files. + +# Verified 2026-09-10: these links deliberately cite canonical source contracts +# and the strict-build policy outside docs/, rather than generated site pages. +# Every target is an existing repository file; retired documents are labeled +# as historical text at their source and are not allowlisted. +WARNING - Doc file 'ARCHITECTURE.md' contains a link '../skills/rpi/references/bounded-adapter.md', but the target is not found among documentation files. +WARNING - Doc file 'agent-workflow-reference.md' contains a link '../skills/memory/SKILL.md', but the target is not found among documentation files. +WARNING - Doc file 'agent-workflow-reference.md' contains a link '../skills/rpi/SKILL.md', but the target is not found among documentation files. +WARNING - Doc file 'agent-workflow-reference.md' contains a link '../skills/validate/references/mechanics.md', but the target is not found among documentation files. +WARNING - Doc file 'how-it-works.md' contains a link '../skills/rpi/SKILL.md', but the target is not found among documentation files. +WARNING - Doc file 'adr/ADR-0016-state-tiers.md' contains a link '../../skills/validate/references/mechanics.md', but the target '../skills/validate/references/mechanics.md' is not found among documentation files. +WARNING - Doc file 'adr/ADR-0017-loop-as-control-flow-not-knowledge.md' contains a link '../../skills/agent-native/references/model-dispatch.md', but the target '../skills/agent-native/references/model-dispatch.md' is not found among documentation files. +WARNING - Doc file 'architecture/hexagon-port-realness-audit.md' contains a link '../../AGENTS.md', but the target '../AGENTS.md' is not found among documentation files. +WARNING - Doc file 'architecture/rpi-traversal.md' contains a link '../../skills/rpi/SKILL.md', but the target '../skills/rpi/SKILL.md' is not found among documentation files. +WARNING - Doc file 'architecture/rpi-traversal.md' contains a link '../../skills/agent-native/references/model-dispatch.md', but the target '../skills/agent-native/references/model-dispatch.md' is not found among documentation files. +WARNING - Doc file 'architecture/rpi-traversal.md' contains a link '../../skills/rpi/references/outer-goal.md', but the target '../skills/rpi/references/outer-goal.md' is not found among documentation files. +WARNING - Doc file 'architecture/rpi-traversal.md' contains a link '../../skills/rpi/references/bounded-adapter.md', but the target '../skills/rpi/references/bounded-adapter.md' is not found among documentation files. +WARNING - Doc file 'architecture/rpi-traversal.md' contains a link '../../skills/memory/SKILL.md', but the target '../skills/memory/SKILL.md' is not found among documentation files. +WARNING - Doc file 'reference/skill-system-evolution.md' contains a link '../../skills/rpi/SKILL.md', but the target '../skills/rpi/SKILL.md' is not found among documentation files. +WARNING - Doc file 'releases/2026-07-30-v3.4.0-notes.md' contains a link '../../skills/using-gc/SKILL.md', but the target '../skills/using-gc/SKILL.md' is not found among documentation files. +WARNING - Doc file 'CONTRIBUTING.md' contains a link '../tests/docs/mkdocs-strict-allowlist.txt', but the target is not found among documentation files. diff --git a/tests/scripts/agentops-product-boundary.bats b/tests/scripts/agentops-product-boundary.bats index 6fe796f5d..78a02f6d1 100644 --- a/tests/scripts/agentops-product-boundary.bats +++ b/tests/scripts/agentops-product-boundary.bats @@ -61,7 +61,7 @@ PY done require_text AGENTS.md \ - "RPI charter -> on-demand Plan -> Implement and checks -> fresh Validate -> finish" + "Accepted intent -> native implementation and checks -> fresh independent judgment -> finish" require_text AGENTS.md \ "Persist machine evidence only for" require_text PRODUCT.md \ diff --git a/tests/scripts/agents-operating-contract.bats b/tests/scripts/agents-operating-contract.bats index 658628318..40ab9e6a5 100644 --- a/tests/scripts/agents-operating-contract.bats +++ b/tests/scripts/agents-operating-contract.bats @@ -1,6 +1,6 @@ #!/usr/bin/env bats -@test "root contract routes the RPI product boundary" { +@test "root contract routes native execution and optional RPI" { repo_root="$(cd "$BATS_TEST_DIRNAME/../.." && pwd)" contract="$repo_root/AGENTS.md" [ -f "$contract" ] @@ -8,8 +8,9 @@ for required in \ "operations layer for agentic engineering" \ "federated integration graph" \ + "zero mandatory AgentOps skills" \ "Lean RPI operating charter" \ - "RPI charter -> on-demand Plan -> Implement and checks -> fresh Validate -> finish" \ + "Accepted intent -> native implementation and checks -> fresh independent judgment -> finish" \ "Persist machine evidence only for a caller request or declared consumer" \ "It owns no aggregate retry controller" \ "fresh independent judgment" \