From af9be7f6632c9647426edb8d4ff1873d5f4a9458 Mon Sep 17 00:00:00 2001 From: hshum Date: Sun, 19 Jul 2026 17:14:16 -0700 Subject: [PATCH] =?UTF-8?q?feat(platform):=20NodeKit=20standard=20tree=20p?= =?UTF-8?q?hase=200=20=E2=80=94=20declarative=20adoption?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adopts the NodeKit Agent Application Standard without moving a single file: - nodekit.yaml: repo identity/ownership/lifecycle + a layout map declaring where each standard concern lives TODAY, with honest conformance flags (forbidVendoredRuntime:false — this harness is the migration source; requireCompiledManifest:false — no compiler exists, no faked .nodeagent/) - nodeagent.yaml: live app composition (authoring dir, convex backend, entity-intelligence pack, four required eval commands that already run) - agent/: filesystem-first authored surface, md/yaml only. instructions.md mirrors chatRuns.ts; policies capture the real approval gates, SSRF blocklist, and budget envelopes with Source: pointers; subagents/ and tools/ are indexes to the code, preventing drifting second copies - packs/entity-intelligence/pack.yaml: capability declared at current paths - docs/architecture/STANDARD_REPO_TREE.md: full slot-by-slot mapping, declared violations, and the staged physical phases (src->apps/web and convex->backend deferred — Vite/Vercel/Convex-CLI constraints noted) Phase-0 rule encoded in agent/README.md: code remains source of truth; no .ts under agent/; nothing imports this surface until the compiler phase. tsc clean; design-system guards green; zero runtime impact. Co-Authored-By: Claude Fable 5 --- agent/README.md | 22 ++++++++ agent/instructions.md | 35 ++++++++++++ agent/policies/approvals.yaml | 21 +++++++ agent/policies/budgets.yaml | 20 +++++++ agent/policies/egress.yaml | 21 +++++++ agent/subagents/README.md | 13 +++++ agent/tools/README.md | 20 +++++++ docs/architecture/STANDARD_REPO_TREE.md | 75 +++++++++++++++++++++++++ nodeagent.yaml | 47 ++++++++++++++++ nodekit.yaml | 47 ++++++++++++++++ packs/entity-intelligence/pack.yaml | 35 ++++++++++++ 11 files changed, 356 insertions(+) create mode 100644 agent/README.md create mode 100644 agent/instructions.md create mode 100644 agent/policies/approvals.yaml create mode 100644 agent/policies/budgets.yaml create mode 100644 agent/policies/egress.yaml create mode 100644 agent/subagents/README.md create mode 100644 agent/tools/README.md create mode 100644 docs/architecture/STANDARD_REPO_TREE.md create mode 100644 nodeagent.yaml create mode 100644 nodekit.yaml create mode 100644 packs/entity-intelligence/pack.yaml diff --git a/agent/README.md b/agent/README.md new file mode 100644 index 00000000..aa3adf43 --- /dev/null +++ b/agent/README.md @@ -0,0 +1,22 @@ +# agent/ — authored agent surface (phase 0) + +This directory is the NodeKit-standard, filesystem-first authoring surface for +the NodeBench agent application. **Phase 0 rule: this surface documents the +live system; the code remains the source of truth.** Nothing here is loaded at +runtime yet — there is no manifest compiler, so we do not pretend there is +(`requireCompiledManifest: false` in `nodekit.yaml`, no fake `.nodeagent/`). + +Every file carries a `Source:` pointer to the code that actually enforces it. +When the `@nodeagent` compiler exists, the direction reverses: these files +become the source and the pointers become generated output. + +| Standard slot | Status | Lives today at | +|---|---|---| +| instructions.md | authored here | mirrors `convex/domains/redesign/chatRuns.ts` | +| policies/ | authored here | mirrors guards in server/, convex/, goals/ | +| subagents/ | index only | `convex/domains/agents/**` orchestrator-workers | +| tools/ | index only | `packages/mcp-local/src/tools/**` (304 tools) | +| planner/ context/ memory/ hooks/ channels/ schedules/ sandbox/ | not yet authored | see `docs/architecture/STANDARD_REPO_TREE.md` mapping | + +Do not add `.ts` files here in phase 0 — the app's tsconfig does not include +this directory and nothing should import from it until the compiler phase. diff --git a/agent/instructions.md b/agent/instructions.md new file mode 100644 index 00000000..7b4b4708 --- /dev/null +++ b/agent/instructions.md @@ -0,0 +1,35 @@ +# NodeBench orchestrator instructions + +Source: `convex/domains/redesign/chatRuns.ts` (system prompts, +`responseShapeSystemInstructions`, `applyDeterministicResponsePolicy`) — +that file is authoritative; this is the authored mirror. + +## Identity + +You are the NodeBench entity-intelligence agent. You synthesize answers about +companies, markets, and questions **with sources**, turn each run into a +reusable artifact (a receipt), and watch for change later. You are not a +chatbot that answers once. + +## Honesty contract (non-negotiable, enforced in code) + +- Never fabricate verification badges, trace steps, telemetry, or scores. + Honest-empty beats fake-full: if a value was not computed, omit it. +- Failures render as failures. A blocked fetch is reported as a blocked fetch, + with the named reason. +- Refusals carry named reasons (`describeCanonicalAnswerFit` reasons array). +- Every factual claim that a reader could act on carries a citation to an + evidence row; sentence-level superlatives are gated on cited evidence. + +## Response shaping + +Detect the requested shape from the user's prompt — title, bullets, sentence, +paragraph, word-limit, JSON, table — and obey it deterministically. The shape +detector and its exhaustive switch live in `chatRuns.ts`; new shapes require a +detector case, a system instruction, AND a guard test. + +## Output anatomy + +Prose leads. Reasoning, tool calls, and sources collapse into the message +(`ChatAssistantMessage` — the ONE canonical renderer for chat surface, receipt +page, and panel). Do not emit memo furniture in conversation. diff --git a/agent/policies/approvals.yaml b/agent/policies/approvals.yaml new file mode 100644 index 00000000..1e531e0d --- /dev/null +++ b/agent/policies/approvals.yaml @@ -0,0 +1,21 @@ +# Human-approval gates. Source: goals/HARD_GATES.md + .claude/rules +# (self_improvement_loop, backend_contract_migration). Enforced by process + +# CI branch protection today, not by a runtime policy engine. +apiVersion: nodeagent.dev/v1 +kind: Policy +metadata: { name: approvals } +spec: + human_required: + - production_deploy # CI-gated PR merge only; never out-of-band convex deploy + - destructive_migration + - auth_changes + - billing_changes + - permission_rule_changes # public/private visibility rules + - data_deletion + - legal_privacy_copy + - external_publish # LinkedIn posts, public submissions + - secrets_management + automatic: + - research_and_docs + - additive_schema_fields # must be v.optional and announced (AGENT_COORDINATION.md) + - ci_gated_pr_merge # gh pr merge --auto --squash diff --git a/agent/policies/budgets.yaml b/agent/policies/budgets.yaml new file mode 100644 index 00000000..c03274e5 --- /dev/null +++ b/agent/policies/budgets.yaml @@ -0,0 +1,20 @@ +# Budget envelopes. Source: server/mcpAuth.ts (gateway limits), server/pipeline +# (judge latency budget), .claude/rules/orchestrator_workers.md (per-subagent caps). +apiVersion: nodeagent.dev/v1 +kind: Policy +metadata: { name: budgets } +spec: + gateway: + rate_limit_per_minute: 100 + idle_timeout_minutes: 30 + close_codes: { auth: 4001, rate_limit: 4002, timeout: 4003 } + run: + judge_latency_budget_ms: 30000 # latencyWithinBudget gate default + timeout_pattern: AbortController + checkpoint gates between async stages + subagent: # every spawn caps all three (orchestrator_workers rule) + wall_clock: bounded + output_tokens: bounded + tool_calls: bounded + retries: + max_per_subagent: 3 + backoff: exponential_with_jitter # 2s / 6s / 18s (+jitter), async_reliability rule diff --git a/agent/policies/egress.yaml b/agent/policies/egress.yaml new file mode 100644 index 00000000..0ab14cf6 --- /dev/null +++ b/agent/policies/egress.yaml @@ -0,0 +1,21 @@ +# Network egress policy. Source: SSRF guards per .claude/rules/agentic_reliability.md +# (checklist item 5), enforced in server fetch paths (URL validation before fetch) +# and bounded reads (item 6). +apiVersion: nodeagent.dev/v1 +kind: Policy +metadata: { name: egress } +spec: + protocols: [https, http] + deny_hosts: # RFC1918 + link-local + metadata — hard blocklist + - localhost + - "127.0.0.0/8" + - "10.0.0.0/8" + - "172.16.0.0/12" + - "192.168.0.0/16" + - "169.254.0.0/16" + - "0.0.0.0" + - "[::1]" + - metadata.google.internal + response_limits: + max_bytes_external_body: 5242880 # BOUND_READ — cancel stream on overflow + credentials_in_url: forbidden diff --git a/agent/subagents/README.md b/agent/subagents/README.md new file mode 100644 index 00000000..5f793142 --- /dev/null +++ b/agent/subagents/README.md @@ -0,0 +1,13 @@ +# subagents/ — index (phase 0) + +The orchestrator-workers implementation lives in `convex/domains/agents/**` +(sub-agent configs, trace types, parallel task trees) per +`.claude/rules/orchestrator_workers.md`: one orchestrator + N sub-agents with +fresh context, each with a scoped task, a tool allowlist, a budget envelope, +and a dedicated scratchpad section. Failure is bounded output + a failure +marker — never silent. + +Standard-shape authoring (`/agent.yaml` + `instructions.md` + +`tools.allow.yaml` + `output.schema`) lands when the runtime consumes authored +definitions. Until then this index prevents a second, drifting copy of the +sub-agent contracts. diff --git a/agent/tools/README.md b/agent/tools/README.md new file mode 100644 index 00000000..96109437 --- /dev/null +++ b/agent/tools/README.md @@ -0,0 +1,20 @@ +# tools/ — index (phase 0) + +The tool surface is `packages/mcp-local/src/tools/**` — 304 tools across 50 +domains, registered in `toolRegistry.ts` (`{name, description, inputSchema, +handler}`), discovered progressively via `discover_tools` / +`get_tool_quick_ref` / `get_workflow_chain`. + +Mapping onto the standard verb taxonomy (for the future authored split): + +| Standard verb | Today's domains (examples) | +|---|---| +| read/ | web fetch/search, entity read, memory/JIT retrieval tools | +| analyze/ | deep-sim, benchmarks, trajectory scoring, judge tools | +| propose/ | drafting, memo/report generation, LinkedIn drafts | +| mutate/ | Convex writes, spreadsheet/document edits, sync tools | +| verify/ | verification cycles, eval harness, watchdog, live checks | + +Conformance note: `toolRegistry.ts` is a hand-maintained global registry — +standard rule 4 ("no manually maintained global tool registry") is a known +violation, resolved by the generated-registry compiler phase, not by hand. diff --git a/docs/architecture/STANDARD_REPO_TREE.md b/docs/architecture/STANDARD_REPO_TREE.md new file mode 100644 index 00000000..7a1517f9 --- /dev/null +++ b/docs/architecture/STANDARD_REPO_TREE.md @@ -0,0 +1,75 @@ +# NodeBench × NodeKit Standard Repo Tree + +Status: **Phase 0 adopted** (declarative — manifests + authored surface, zero +file moves). This doc is the mapping between the NodeKit Agent Application +Standard tree and where each concern lives in this repo, plus the staged plan +for physical conformance. + +Prior art: NodeKit Agent Application Standard (node-platform); Eve +filesystem-first authoring (`apps/eve-agent/agent/`); 2026-07-19 harness audit +of eve / NodeAgent / NodeRoom. + +## Phase 0 (this PR) — declarative adoption + +Added, all additive and runtime-inert: + +``` +nodekit.yaml repo identity, ownership, lifecycle, layout map, honest conformance flags +nodeagent.yaml app composition: authoring dir, runtime, backend, pack, required evals +agent/ authored surface (md/yaml only; code stays source of truth) +├── README.md phase-0 rules — no fake compiled output, Source: pointers mandatory +├── instructions.md orchestrator instructions (mirrors chatRuns.ts) +├── policies/{approvals,egress,budgets}.yaml +├── subagents/README.md index → convex/domains/agents orchestrator-workers +└── tools/README.md index → packages/mcp-local (304 tools) + verb-taxonomy map +packs/entity-intelligence/pack.yaml capability declared at current locations +``` + +Explicit non-goals of phase 0: no `.nodeagent/` (no compiler exists — we do not +fake generated output), no `.ts` under `agent/` (tsconfig excludes it; nothing +may import it), no file moves. + +## Full mapping (standard slot → today) + +Legend: `✓` conformant · `◐` exists at non-standard location · `✗` missing + +| Standard | State | Today | +|---|---|---| +| `nodekit.yaml` / `nodeagent.yaml` | ✓ (phase 0) | repo root | +| `AGENTS.md`, `README.md`, `components.json` | ✓ | repo root | +| `.claude/` agents·skills·rules | ✓ | richest in the family | +| `agent/` authored surface | ◐ | phase-0 mirror; truth in `convex/domains/redesign/chatRuns.ts`, `convex/domains/agents/**`, `server/pipeline/**` | +| `packs/` | ◐ | `packs/entity-intelligence/pack.yaml` declares content spread across convex/, packages/, server/ | +| `integrations/pi-ai` | ✗ | provider seam = `shared/llm/modelCatalog.ts` direct | +| `apps/web/src` | ◐ | `src/` at root (Vite root) | +| `backend/convex` | ◐ | `convex/` at root (Convex CLI default) | +| `evals/{benchmark,conformance,production,smoke}` | ◐ | `packages/mcp-local/src/benchmarks`, `tests/e2e`, `scripts/verify-live.ts`, dogfood scripts | +| `proof/` | ◐ | `docs/design/ui-contract/` (evidence folders + `manifest.schema.json`) | +| `adw/` | ◐ | `scripts/improvement-loop/` + `goals/` (+ `HARD_GATES.md`) | +| `fixtures/` | ◐ | spread through test dirs; deterministic fixtures exist | +| `workers/` | ◐ | `server/` (voice, gateway) | +| `.nodeagent/` generated | ✗ | blocked on the manifest compiler (deliberate) | +| `packages/mcp-local` etc. | ★ keep | NodeBench-specific distribution artifact; standard-compatible | + +## Known conformance violations (declared, not hidden) + +1. **Repo-local runtime** — this repo's harness predates NodeAgent and is a + *migration source*, not a vendored copy (`forbidVendoredRuntime: false`). + Flips after the ecosystem phase-2 extraction (NodeRoom runtime → NodeAgent) + and this repo consumes `@nodeagent/runtime`. +2. **Hand-maintained global tool registry** (`toolRegistry.ts`) — standard + rule 4; resolved by the generated-registry compiler, not by hand. +3. **modelCatalog duplication** — copies exist in NodeRoom and NodeAgent; + resolved by `@nodeagent/providers` extraction. + +## Physical phases (each gated, none started) + +| Phase | Move | Constraint to respect | +|---|---|---| +| 1 | `evals/`, `proof/`, `fixtures/`, `adw/` consolidation | script/CI path updates only — low risk, do first | +| 2 | `src/` → `apps/web/src`; `convex/` → `backend/convex` | Vite root + Vercel config + tsconfig paths + `convex.json` functions path; import churn across hundreds of files; MUST follow expand-contract discipline and land between deploys | +| 3 | `agent/` becomes compiled truth; `.nodeagent/` generated | blocked on `@nodeagent` manifest compiler | +| 4 | pack extraction (`packs/entity-intelligence` gains real content) | mirrors NodeSlide layering: core/pack/adapter/react split | + +Rule for all phases: **build every new capability against the standard +locations from now on**; never move files and change behavior in the same PR. diff --git a/nodeagent.yaml b/nodeagent.yaml new file mode 100644 index 00000000..3bda337d --- /dev/null +++ b/nodeagent.yaml @@ -0,0 +1,47 @@ +# NodeAgent application composition — phase 0 (declares the LIVE system). +# The runtime engine is repo-local today; "engine: nodeagent-native" becomes +# accurate only after the NodeRoom->NodeAgent runtime extraction lands and this +# repo consumes @nodeagent/runtime. Until then this manifest documents reality. +apiVersion: nodeagent.dev/v1 +kind: AgentApplication + +metadata: + name: nodebench-entity-intelligence + version: 1.0.0 + +spec: + authoring: + directory: ./agent + # Source of truth remains code until a manifest compiler exists: + # convex/domains/redesign/chatRuns.ts (orchestrator prompts + response shapes) + # convex/domains/agents/** (orchestrator-workers, trace types) + # server/pipeline/** (judge gates, projection writer) + + runtime: + engine: repo-local # target: nodeagent-native + profile: production + + backend: + adapter: convex + path: ./convex + + packs: + - ./packs/entity-intelligence/pack.yaml + + orchestration: + planner: orchestrator-workers # .claude/rules/orchestrator_workers.md + maxConcurrentSubagents: 10 # harness lane cap + + policies: + ref: ./agent/policies + + evaluations: + required: + - id: conformance + command: npm run test:e2e:check # ui-contract runner (Tier B per PR) + - id: smoke + command: npm run dogfood:verify:smoke + - id: benchmark + command: npx tsx packages/mcp-local/src/benchmarks/searchQualityEval.ts + - id: production + command: npm run verify-live diff --git a/nodekit.yaml b/nodekit.yaml new file mode 100644 index 00000000..8e89d10c --- /dev/null +++ b/nodekit.yaml @@ -0,0 +1,47 @@ +# NodeKit repository contract — phase 0 (declarative adoption, no file moves). +# Standard: NodeKit Agent Application Standard (node-platform). +# Honest posture: this repo predates the standard; several concerns live at +# legacy paths. Each is declared where it IS, not where the standard wants it. +apiVersion: nodekit.dev/v1 +kind: Repository + +metadata: + name: nodebench-ai + github: HomenShum/NodeBenchAI + role: domain-application # registry currently says legacy-product; this manifest is the correction proposal + status: production # live at https://www.nodebenchai.com, actively shipped + +spec: + ownership: + runtime: NodeAgent # target owner; today the harness is repo-local (see conformance notes) + trace: NodeTrace + memory: NodeMem + proof: NodeProof + + lifecycle: + bootstrap: npm install + demo: npm run dev + check: npm run test:run + proof: npm run verify-live + + # Where standard-tree concerns actually live today (migration map). + layout: + agentAuthoring: ./agent # phase-0 authored surface; compiled truth remains in code + packs: + - ./packs/entity-intelligence/pack.yaml + backend: ./convex # Convex CLI expects repo-root convex/; move deferred (phase 2) + web: ./src # Vite root; apps/web split deferred (phase 2) + evals: + benchmark: packages/mcp-local/src/benchmarks + conformance: tests/e2e # ui-contract-runner + surface contracts + production: scripts/verify-live.ts + proof: docs/design/ui-contract # evidence folders + manifest.schema.json + adw: scripts/improvement-loop # + goals/ (goal cards, HARD_GATES.md) + mcpDistribution: packages/mcp-local # 304-tool MCP server (npx nodebench-mcp) + + conformance: + forbidVendoredRuntime: false # repo-local harness is the MIGRATION SOURCE, not a vendored copy; + # flips to true after @nodeagent/runtime extraction (phase 2 of + # the ecosystem migration) and this repo consumes the package. + requireCompiledManifest: false # no .nodeagent/ compiler exists yet; do not fake generated output + requireProductionProof: true # scripts/verify-live.ts (Tier A) + npm run live-smoke (Tier B) diff --git a/packs/entity-intelligence/pack.yaml b/packs/entity-intelligence/pack.yaml new file mode 100644 index 00000000..22d55a37 --- /dev/null +++ b/packs/entity-intelligence/pack.yaml @@ -0,0 +1,35 @@ +# Entity-intelligence capability pack — phase 0 declaration. +# Declares the capability where it LIVES; extraction into a portable pack +# directory is a later phase (mirrors the NodeSlide layering decision). +apiVersion: nodeagent.dev/v1 +kind: CapabilityPack + +metadata: + name: entity-intelligence + version: 0.1.0 + description: > + Company/market/question intelligence: sourced synthesis, 8-section entity + workspace, diligence blocks, daily brief + narrative workflows, forecasting, + reusable run receipts, change watching. + +spec: + locations: # current sources of truth (not yet extracted) + tools: + - packages/mcp-local/src/tools # enrichment, web, memory, deep-sim domains + orchestration: + - convex/domains/redesign/chatRuns.ts # run pipeline + response policy + - convex/workflows # dailyLinkedInPost, narrative + validators: + - server/routes/search.ts # 4-layer grounding (confidence, isGrounded, citations) + - server/pipeline # diligence judge, 10-gate catalog + artifacts: + - convex/domains/redesign # chat runs as receipts (mintShowcaseRun) + - docs/design/ui-contract # UI evidence manifests + renderers: + - src/features/redesign/components/ChatAssistantMessage.tsx # THE canonical answer + evals: + benchmark: packages/mcp-local/src/benchmarks/searchQualityEval.ts # 103 queries, 18 categories + conformance: tests/e2e/ui-contract-runner.spec.ts + production: scripts/verify-live.ts + fixtures: + deterministic: true # demo conversations + QA answer seed (?qaState=answer)