From 3618be85d8c83bcae5e0d1d148e264d9a3775641 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Eduardo=20Javier=20Garc=C3=ADa=20L=C3=B3pez?= <6173685+LaloLalo1999@users.noreply.github.com> Date: Fri, 28 Aug 2026 15:08:39 -0600 Subject: [PATCH] docs: map orchestration evidence boundary --- README.md | 1 + docs/portfolio-integration.md | 48 +++++++++++++++++++++++++++++++++++ 2 files changed, 49 insertions(+) create mode 100644 docs/portfolio-integration.md diff --git a/README.md b/README.md index f56bdcc..f2d98a2 100644 --- a/README.md +++ b/README.md @@ -102,6 +102,7 @@ Authoritative PRD/design files live under: ## Documentation map - **Agent guidelines**: `AGENTS.md` — conventions, commands, and code patterns for this repo. +- **Portfolio integration**: `docs/portfolio-integration.md` — experiment status, evidence boundary, and future adapter seam. - **Runbooks**: `docs/runbooks/local-dev.md`, `docs/runbooks/ci.md`, `docs/runbooks/deployment.md`, `docs/runbooks/posthog.md`, `docs/runbooks/rollback.md`, `docs/runbooks/secrets.md`, `docs/runbooks/workflows-workpools.md` - **ADRs**: `docs/adr/0001-agent-sandbox.md` through `docs/adr/0006-convex-components-billing-orchestration.md` - **Product spec**: `docs/product/agent_company_sim_detailed_spec.md` diff --git a/docs/portfolio-integration.md b/docs/portfolio-integration.md new file mode 100644 index 0000000..884a5f9 --- /dev/null +++ b/docs/portfolio-integration.md @@ -0,0 +1,48 @@ +# Portfolio integration contract + +This document maps Code Colony into the delivery portfolio without treating the seeded simulation +as live orchestration evidence. It does not define the root contract, and no Ultimate Harness or +Telar adapter is shipped here. + +## Role and maturity + +| Classification | Current status | +|---|---| +| Primary role | Orchestration experiment and operator-experience prototype | +| Real surfaces | Application shells, domain types, selected Convex schema/components, Mastra triage agent, infrastructure, and PostHog boundary | +| Simulated surfaces | OMP runtime execution, diffs/checks/cost ledger, artifact review, memory/skill promotion, presence/votes, and marketplace behavior | +| Current evidence status | Not applicable for production runtime or benchmark claims | +| Portfolio integration | Proposed; no Ultimate Harness or Telar adapter is shipped here | + +## Evidence boundary + +| Category | Current system of record | Allowed claim | +|---|---|---| +| Product behavior | Source, tests, built app, and exercised rendered path | The verified prototype interaction only | +| Seeded orchestration | `src/domain/types.ts` and seed transitions | UI/domain simulation, never a completed external run | +| Convex state | Owning schema and wrapper functions | Implemented backend structure; not proof that every frontend flow is wired | +| Runtime/model receipt | None for the simulated OMP run | `unavailable` | +| Quality/cost/latency/tokens | Mock values where labeled; real telemetry only when a live path emits it | Never mix mock and observed fields in one receipt | +| Blind review | Not implemented | No independent/blind-quality claim | + +PostHog is the planned/implemented telemetry seam for real product and runtime events, but the +existence of an instrumentation module does not prove an event was emitted or received. + +## Portfolio seam + +1. Code Colony owns the simulation and operator experience. It must keep mock data visibly labeled. +2. A future live runtime integration consumes a small adapter interface and returns immutable run, + artifact, and verification references. It must not leak provider internals into UI state. +3. Ultimate Harness may later remain the only Run Control for externally executed attempts. Code + Colony visualizes its receipts; it does not start a second controller for the same attempt. +4. The proposed Telar plane may own planning/governance and normalized evidence references. No + current Code Colony screen is proof that Telar behavior exists. + +## Future plan + +- Replace the simulated runtime card only after a real adapter produces attempt lineage, resolved + harness/model, repository/config hashes, artifact hashes, checks, cost, latency, tokens, and + explicit missing-field reasons. +- Keep independent review separate from implementation and retain judge/reviewer identity, + evidence view, revision, verdict, and limitations. +- Prove one end-to-end spec-to-diff path before exposing portfolio rollups or learning promotion.