From b65173d0bfc301ad2603ef700b19fc369c22f56a Mon Sep 17 00:00:00 2001 From: ZuluYokohama Date: Sat, 6 Jun 2026 00:49:44 -0500 Subject: [PATCH 01/10] docs: add truth-resolver prototype design spec Hybrid truth-resolver gate (SP2, prototype-first): distill -> overlap (intent/context/value) -> score_value -> Verdict, with the semantic distillation step behind a pluggable Distiller seam (deterministic now, LLM later). Encodes knowledge-integration as a value dimension. Runs and unit-tests with no API key. Doctrine (SP1) and operating-mode binding (SP3) are deferred cycles. Co-Authored-By: Claude Opus 4.8 (1M context) --- ...6-06-06-truth-resolver-prototype-design.md | 151 ++++++++++++++++++ 1 file changed, 151 insertions(+) create mode 100644 docs/superpowers/specs/2026-06-06-truth-resolver-prototype-design.md diff --git a/docs/superpowers/specs/2026-06-06-truth-resolver-prototype-design.md b/docs/superpowers/specs/2026-06-06-truth-resolver-prototype-design.md new file mode 100644 index 0000000..18717ca --- /dev/null +++ b/docs/superpowers/specs/2026-06-06-truth-resolver-prototype-design.md @@ -0,0 +1,151 @@ +# Truth-Resolver Prototype — Design Spec + +> **Status:** Approved for planning (2026-06-06) +> **Author:** ZuluYokohama (b.jones@jtech.ai), via Claude Code brainstorming +> **Scope:** Sub-project SP2 (the Module), built **prototype-first**. SP1 (doctrine doc) is +> back-filled from what the prototype proves; SP3 (agent operating-mode binding) is a later cycle. + +## 1. Purpose + +A standalone gate that, given a proposed **action** and the **actors** involved +(user / agent / subagent / element), returns a **Verdict**: is the action *aligned* across the +actors' distilled **intent**, **context**, and **value**, and is it **net-positive on value**? + +It operationalises the repo's `[ INTENT ] ≡ [ CODE OPS ] ≡ [ VALUE GEN ]` doctrine as a runnable +resolver. Unlike the existing deterministic gates (`aaa_quality`, `evolution_gate`: flake8 / +bandit / radon / pytest), this gate scores **semantic alignment** — so its one genuinely-semantic +step is quarantined behind a pluggable interface (the "seam"), letting the prototype run and +unit-test deterministically today while leaving a drop-in upgrade to an LLM distiller later. + +**Prototype boundary:** standalone callable + CLI (`python scripts/truth_resolver.py action.json`) +and an importable `resolve(...)`. It does **not** wire into the live evolution loop, call any LLM, +or touch the in-flight R3F work. + +## 2. Core model + +Each actor contributes an `AlignmentVector { intent, context, value }`. The pipeline isolates the +semantic step to `Distiller.distill`: + +``` +Distiller.distill(raw_actor) -> AlignmentVector # ← THE SEAM (deterministic now, LLM later) +overlap(vectors) -> per-dimension alignment 0..1 # deterministic (Jaccard) +score_value(action, vectors) -> net value (float) # deterministic (weighted delta) +resolve(alignment, value) -> Verdict +``` + +- **`Distiller`** is an interface (Python `Protocol`/ABC). `DeterministicDistiller` (this prototype): + normalises free text into lowercased token/tag sets per dimension. `LLMDistiller` (future, NOT in + this prototype): same interface, calls a model to distill normalized intent/context/value. +- **`overlap`** computes pairwise + aggregate alignment of `intent`, `context`, and `value` token + sets across actors using Jaccard similarity (|A∩B| / |A∪B|), returning a 0..1 score per dimension + and an aggregate (mean of dimension scores). +- **`score_value`** scores the action's net value as a weighted sum over value dimensions + (see §4), where each dimension's contribution is the action's claimed/realised delta on that + dimension, gated by whether the actors share that value (no shared value → no credit). + +## 3. The Verdict (output) + +```python +@dataclass +class Verdict: + passed: bool # alignment >= ALIGN_THRESHOLD AND value > 0.0 + alignment: float # 0..1 aggregate overlap across actors + value: float # net value score (may be negative) + rationale: str # human-readable explanation of the decision + dissent: list[str] # actor/dimension pairs that broke alignment, e.g. "subagent:context" +``` + +`dissent` is the load-bearing output: it names *where* truth failed to resolve so a misalignment is +actionable, not just a red light. `ALIGN_THRESHOLD` is a module constant (default `0.5`), +overridable per call. + +## 4. "Value, dimensioned for knowledge integration" + +`score_value` evaluates the action across explicit **value dimensions**: + +- `correctness`, `performance`, `security` — mirroring the existing gate family. +- **`knowledge_integration`** — the dimension named in the original intent. Scores how much the + action *coheres* with existing knowledge: reuses existing abstractions, reduces drift, and + preserves the `[INTENT]≡[CODE OPS]≡[VALUE GEN]` isomorphism, vs fragments it. Concretely, the + prototype derives this from the action's `value_claims` (e.g. `"no new deps"`, `"reuses X"`, + `"DRY"` raise it; `"new dependency"`, `"duplicate"`, `"fork-specific"` lower it) intersected with + the actors' shared values. This structurally **rewards DRY, isomorphic, low-drift actions** — + encoding the repo's own doctrine as a scoring dimension. + +Dimension weights are a module-level dict (default equal weights), overridable per call. + +## 5. Inputs + +A JSON document describing the action and actors: + +```json +{ + "action": { + "summary": "Tunnel intent form to 3D pulse via a hand-rolled store", + "value_claims": ["no new deps", "reuses useSyncExternalStore", "O(1)"] + }, + "actors": [ + {"role": "user", "intent": "...", "context": "...", "values": ["correctness", "knowledge_integration"]}, + {"role": "agent", "intent": "...", "context": "...", "values": ["correctness", "performance"]}, + {"role": "subagent", "intent": "...", "context": "...", "values": ["correctness"]} + ] +} +``` + +`role`, `intent`, `context` are strings; `values` is a list of value-dimension names. The CLI reads +this from a file path argument; `resolve()` accepts the parsed dict (or typed objects). + +## 6. File structure + +- `scripts/truth_resolver.py` — dataclasses (`AlignmentVector`, `Verdict`), the `Distiller` + interface + `DeterministicDistiller`, the pure functions (`overlap`, `score_value`, `resolve`), + and a `main()` CLI entry. One file, one responsibility (resolve truth for an action); kept under + ~200 lines. If it grows past that, split `distillers.py` out — but not preemptively (YAGNI). +- `tests/test_truth_resolver.py` — unit tests (see §7). + +No changes to any existing file in this prototype. + +## 7. Testing (TDD, deterministic) + +Red/green unit tests for every pure piece: + +- `overlap`: identical actors → 1.0; fully disjoint → 0.0; partial → exact Jaccard value; + per-dimension breakdown correct. +- `score_value`: net-positive claims vs shared values → positive; value claimed but unshared by + actors → no credit; drift-claims (`"new dependency"`) → negative contribution. +- `resolve`: alignment ≥ threshold AND value > 0 → `passed=True`; either failing → `False`, with + correct `dissent` entries naming the actor:dimension that broke alignment. +- `DeterministicDistiller`: text → normalized token/tag sets (lowercase, punctuation-stripped, + known value-tags mapped). +- `LLMDistiller` is **not** implemented here; a test asserts the `Distiller` interface is satisfiable + (a trivial stub conforms) so the seam is real. + +Runs green with no API key, like the other gates. `pytest -q` is the runner. + +## 8. Composition & future cycles (out of scope here) + +- **SP1 (doctrine doc):** `docs/TRUTH_RESOLVER.md`, a sibling to `ISOMORPHISM_MANIFESTO.md`, + back-filled from the prototype's proven model. Later cycle. +- **SP2-proper (live integration):** wire `resolve()` into the evolution/AAA pre-gate as an + alignment pre-check. Later cycle. +- **SP3 (agent operating-mode):** bind the orchestrator's autonomous "fix-for-value" behaviour to a + `resolve()` alignment check (the safe form of the perpetual-permission grant) via a memory/feedback + preference and optional hooks. Later cycle. **Irreversible/outward-facing actions remain + human-gated regardless of resolver verdict.** + +## 9. Definition of Done (prototype) + +- `scripts/truth_resolver.py` implements §2–§5 with the pluggable `Distiller` seam. +- `pytest -q tests/test_truth_resolver.py` is green (no API key). +- CLI runs against a sample `action.json` and prints a Verdict (pass + fail examples). +- No existing file modified; no LLM call; R3F work untouched. + +## 10. Risks & constraints + +- **Semantic fidelity:** the deterministic distiller reduces "distillation" to token/tag overlap — + honest about this; the seam exists precisely so the real semantics arrive via `LLMDistiller` + without a rewrite. The prototype proves the *pipeline and contract*, not production-grade alignment. +- **Value scoring is heuristic:** `score_value` over `value_claims` is a deliberately simple, + inspectable heuristic for the prototype, not a learned model. Documented as such. +- **Determinism:** no `Date.now()`/random; all scoring is pure functions of inputs, so tests are + exact. From e1aa4bc6712c7461bfb9ac220f4416db3edc7273 Mon Sep 17 00:00:00 2001 From: ZuluYokohama Date: Sat, 6 Jun 2026 00:54:38 -0500 Subject: [PATCH 02/10] docs: add truth-resolver prototype implementation plan MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit TDD plan (pytest) for scripts/truth_resolver.py: 6 tasks building dataclasses + Distiller seam, DeterministicDistiller, Jaccard overlap, shared-value-gated value scoring, resolve()+dissent, and a CLI — 23 deterministic tests, no API key, LLMDistiller left as a proven seam. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../2026-06-06-truth-resolver-prototype.md | 716 ++++++++++++++++++ 1 file changed, 716 insertions(+) create mode 100644 docs/superpowers/plans/2026-06-06-truth-resolver-prototype.md diff --git a/docs/superpowers/plans/2026-06-06-truth-resolver-prototype.md b/docs/superpowers/plans/2026-06-06-truth-resolver-prototype.md new file mode 100644 index 0000000..21828de --- /dev/null +++ b/docs/superpowers/plans/2026-06-06-truth-resolver-prototype.md @@ -0,0 +1,716 @@ +# Truth-Resolver Prototype Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Build a deterministic, unit-tested `truth_resolver.py` prototype gate that scores whether a proposed action is aligned (across actors' intent/context/value) and net-positive on value, with the one semantic step isolated behind a pluggable `Distiller` seam. + +**Architecture:** Pure-function pipeline `distill → overlap → score_value → resolve`. The only swappable part is `Distiller` (a `runtime_checkable` Protocol); the prototype ships `DeterministicDistiller` (text → token/tag sets) and intentionally does NOT implement `LLMDistiller` (a test only asserts the interface is satisfiable). All scoring is pure functions of inputs — no `Date`/random, no LLM, no network — so tests are exact. + +**Tech Stack:** Python 3.14, stdlib only (`dataclasses`, `typing.Protocol`, `re`, `json`, `sys`), `pytest` as the test runner. Follows the repo convention: tests `sys.path`-append `../scripts` and import modules by bare name. + +--- + +## File Structure + +- **Create** `scripts/truth_resolver.py` — the whole prototype (dataclasses, `Distiller` Protocol + `DeterministicDistiller`, pure functions `overlap`/`score_value`/`resolve`, helpers, `main()` CLI). One responsibility: resolve truth for an action. Target < 200 lines. +- **Create** `tests/test_truth_resolver.py` — pytest tests, grown one task at a time. +- **Create** `tests/fixtures/action_pass.json`, `tests/fixtures/action_fail.json` — CLI fixtures (Task 6). +- **Modify:** none. No existing file is touched; nothing is wired into the evolution loop. + +**Test file header** (created in Task 1, unchanged after): every test references the module as `tr.`, so a not-yet-written function fails with `AttributeError` (RED) without breaking the file import. + +```python +import os +import sys +import json +import subprocess + +sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..', 'scripts'))) +import truth_resolver as tr # noqa: E402 +``` + +--- + +## Task 0: Ensure pytest is available (one-time) + +- [ ] **Step 1: Check/instal pytest** + +Run: `python -m pytest --version` +If it errors with "No module named pytest", run: `python -m pip install --quiet --disable-pip-version-check pytest` +Expected: a version prints (e.g. `pytest 8.x`). + +(No commit — environment setup only.) + +--- + +## Task 1: Module scaffold — constants, dataclasses, `Distiller` protocol + +**Files:** +- Create: `scripts/truth_resolver.py` +- Create: `tests/test_truth_resolver.py` + +- [ ] **Step 1: Write the failing test** + +Create `tests/test_truth_resolver.py` with the header above, then append: + +```python +def test_constants_and_dataclasses_exist(): + assert tr.ALIGN_THRESHOLD == 0.5 + assert tr.VALUE_DIMENSIONS == ("correctness", "performance", "security", "knowledge_integration") + + v = tr.AlignmentVector(role="user", intent={"a"}, context={"b"}, value={"correctness"}) + assert v.role == "user" + assert v.intent == {"a"} and v.context == {"b"} and v.value == {"correctness"} + + verdict = tr.Verdict(passed=True, alignment=1.0, value=2.0, rationale="ok", dissent=[]) + assert verdict.passed is True and verdict.alignment == 1.0 + assert verdict.value == 2.0 and verdict.rationale == "ok" and verdict.dissent == [] + + +def test_distiller_is_runtime_checkable_protocol(): + class StubDistiller: + def distill(self, actor): + return tr.AlignmentVector(role="x", intent=set(), context=set(), value=set()) + assert isinstance(StubDistiller(), tr.Distiller) + assert not isinstance(object(), tr.Distiller) +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `python -m pytest tests/test_truth_resolver.py -q` +Expected: FAIL — `ModuleNotFoundError: No module named 'truth_resolver'`. + +- [ ] **Step 3: Write minimal implementation** + +Create `scripts/truth_resolver.py`: + +```python +"""Truth-Resolver prototype gate. + +Resolves whether a proposed action is aligned across the involved actors' +intent / context / value, and net-positive on value. Deterministic pipeline +with the single semantic step (distillation) isolated behind the `Distiller` +interface so an LLM-backed distiller can drop in later without a rewrite. +""" +from __future__ import annotations + +from dataclasses import dataclass +from typing import Protocol, runtime_checkable + +ALIGN_THRESHOLD = 0.5 +VALUE_DIMENSIONS = ("correctness", "performance", "security", "knowledge_integration") +DEFAULT_WEIGHTS = {d: 1.0 for d in VALUE_DIMENSIONS} + + +@dataclass +class AlignmentVector: + """One actor's distilled stance, as token/tag sets per dimension.""" + role: str + intent: set[str] + context: set[str] + value: set[str] + + +@dataclass +class Verdict: + """The resolver's decision for an action.""" + passed: bool + alignment: float + value: float + rationale: str + dissent: list[str] + + +@runtime_checkable +class Distiller(Protocol): + """The pluggable seam: turn a raw actor dict into an AlignmentVector.""" + def distill(self, actor: dict) -> AlignmentVector: ... +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `python -m pytest tests/test_truth_resolver.py -q` +Expected: PASS (2 tests). + +- [ ] **Step 5: Commit** + +```bash +git add scripts/truth_resolver.py tests/test_truth_resolver.py +git commit -m "feat(truth-resolver): scaffold dataclasses + Distiller protocol" +``` + +--- + +## Task 2: `DeterministicDistiller` (text → token/tag sets) + +**Files:** +- Modify: `scripts/truth_resolver.py` +- Test: `tests/test_truth_resolver.py` + +- [ ] **Step 1: Write the failing test** + +Append to `tests/test_truth_resolver.py`: + +```python +def test_tokens_splits_lowercases_and_drops_empties(): + assert tr._tokens("O(1) memory, Zero-Drift!") == {"o", "1", "memory", "zero", "drift"} + assert tr._tokens("") == set() + + +def test_deterministic_distiller_builds_vector(): + d = tr.DeterministicDistiller() + actor = { + "role": "user", + "intent": "Reduce drift to O(1)", + "context": "studio build", + "values": ["Correctness", "Knowledge_Integration"], + } + v = d.distill(actor) + assert isinstance(v, tr.AlignmentVector) + assert v.role == "user" + assert v.intent == {"reduce", "drift", "to", "o", "1"} + assert v.context == {"studio", "build"} + assert v.value == {"correctness", "knowledge_integration"} # lowercased + + +def test_deterministic_distiller_defaults_for_missing_fields(): + v = tr.DeterministicDistiller().distill({}) + assert v.role == "unknown" + assert v.intent == set() and v.context == set() and v.value == set() +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `python -m pytest tests/test_truth_resolver.py -q -k distiller or tokens` +Expected: FAIL — `AttributeError: module 'truth_resolver' has no attribute '_tokens'`. + +- [ ] **Step 3: Write minimal implementation** + +Add to `scripts/truth_resolver.py` (after the `Distiller` protocol, add the `re` import at the top with the other imports): + +```python +import re +``` + +```python +def _tokens(text: str) -> set[str]: + """Lowercase, split on non-alphanumerics, drop empties.""" + return {t for t in re.split(r"[^a-z0-9]+", text.lower()) if t} + + +class DeterministicDistiller: + """Default distiller: normalize free text into token/tag sets. No LLM.""" + def distill(self, actor: dict) -> AlignmentVector: + return AlignmentVector( + role=actor.get("role", "unknown"), + intent=_tokens(actor.get("intent", "")), + context=_tokens(actor.get("context", "")), + value={v.strip().lower() for v in actor.get("values", [])}, + ) +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `python -m pytest tests/test_truth_resolver.py -q` +Expected: PASS (5 tests total). + +- [ ] **Step 5: Commit** + +```bash +git add scripts/truth_resolver.py tests/test_truth_resolver.py +git commit -m "feat(truth-resolver): add DeterministicDistiller + tokenizer" +``` + +--- + +## Task 3: `overlap` (Jaccard per dimension + aggregate) + +**Files:** +- Modify: `scripts/truth_resolver.py` +- Test: `tests/test_truth_resolver.py` + +- [ ] **Step 1: Write the failing test** + +Append: + +```python +def _vec(role, intent, context, value): + return tr.AlignmentVector(role=role, intent=set(intent), context=set(context), value=set(value)) + + +def test_overlap_identical_actors_is_one(): + a = _vec("user", ["x", "y"], ["c"], ["correctness"]) + b = _vec("agent", ["x", "y"], ["c"], ["correctness"]) + ov = tr.overlap([a, b]) + assert ov["dimensions"] == {"intent": 1.0, "context": 1.0, "value": 1.0} + assert ov["aggregate"] == 1.0 + + +def test_overlap_disjoint_actors_is_zero(): + a = _vec("user", ["x"], ["c1"], ["correctness"]) + b = _vec("agent", ["y"], ["c2"], ["performance"]) + ov = tr.overlap([a, b]) + assert ov["dimensions"] == {"intent": 0.0, "context": 0.0, "value": 0.0} + assert ov["aggregate"] == 0.0 + + +def test_overlap_partial_is_exact_jaccard(): + # intent: {x,y} vs {y,z} -> inter {y}=1, union {x,y,z}=3 -> 1/3 + a = _vec("user", ["x", "y"], ["c"], ["correctness"]) + b = _vec("agent", ["y", "z"], ["c"], ["correctness"]) + ov = tr.overlap([a, b]) + assert abs(ov["dimensions"]["intent"] - (1 / 3)) < 1e-9 + assert ov["dimensions"]["context"] == 1.0 + assert ov["dimensions"]["value"] == 1.0 + + +def test_overlap_all_empty_dimension_is_vacuously_one(): + a = _vec("user", [], [], []) + b = _vec("agent", [], [], []) + ov = tr.overlap([a, b]) + assert ov["aggregate"] == 1.0 +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `python -m pytest tests/test_truth_resolver.py -q -k overlap` +Expected: FAIL — `AttributeError: ... has no attribute 'overlap'`. + +- [ ] **Step 3: Write minimal implementation** + +Add to `scripts/truth_resolver.py`: + +```python +def _jaccard_all(sets: list[set[str]]) -> float: + """Multi-set Jaccard: |intersection of all| / |union of all|. + + Empty union (every actor empty on this dimension) is vacuously aligned -> 1.0. + """ + if not sets: + return 1.0 + union: set[str] = set().union(*sets) + if not union: + return 1.0 + inter = set(sets[0]).intersection(*sets[1:]) + return len(inter) / len(union) + + +def overlap(vectors: list[AlignmentVector]) -> dict: + """Per-dimension and aggregate alignment across actors.""" + dims = { + "intent": _jaccard_all([v.intent for v in vectors]), + "context": _jaccard_all([v.context for v in vectors]), + "value": _jaccard_all([v.value for v in vectors]), + } + aggregate = sum(dims.values()) / len(dims) + return {"dimensions": dims, "aggregate": aggregate} +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `python -m pytest tests/test_truth_resolver.py -q` +Expected: PASS (9 tests total). + +- [ ] **Step 5: Commit** + +```bash +git add scripts/truth_resolver.py tests/test_truth_resolver.py +git commit -m "feat(truth-resolver): add per-dimension Jaccard overlap" +``` + +--- + +## Task 4: `score_value` (claim → dimension, gated by shared values) + +**Files:** +- Modify: `scripts/truth_resolver.py` +- Test: `tests/test_truth_resolver.py` + +- [ ] **Step 1: Write the failing test** + +Append: + +```python +def test_score_value_credits_positive_claim_when_dimension_shared(): + a = _vec("user", ["i"], ["c"], ["knowledge_integration"]) + b = _vec("agent", ["i"], ["c"], ["knowledge_integration"]) + action = {"value_claims": ["no new deps", "reuses"]} + # both claims map to knowledge_integration (+1 each), dim shared -> +2.0 + assert tr.score_value(action, [a, b]) == 2.0 + + +def test_score_value_no_credit_when_dimension_not_shared(): + a = _vec("user", ["i"], ["c"], ["correctness"]) + b = _vec("agent", ["i"], ["c"], ["performance"]) # no shared value dim + action = {"value_claims": ["no new deps"]} + assert tr.score_value(action, [a, b]) == 0.0 + + +def test_score_value_negative_claims_subtract(): + a = _vec("user", ["i"], ["c"], ["knowledge_integration"]) + b = _vec("agent", ["i"], ["c"], ["knowledge_integration"]) + action = {"value_claims": ["new dependency", "duplicate"]} + assert tr.score_value(action, [a, b]) == -2.0 + + +def test_score_value_maps_claims_to_correct_dimensions(): + a = _vec("user", ["i"], ["c"], ["performance", "correctness"]) + b = _vec("agent", ["i"], ["c"], ["performance", "correctness"]) + action = {"value_claims": ["o(1)", "tested"]} # performance +1, correctness +1 + assert tr.score_value(action, [a, b]) == 2.0 + + +def test_score_value_ignores_unknown_claims(): + a = _vec("user", ["i"], ["c"], ["knowledge_integration"]) + b = _vec("agent", ["i"], ["c"], ["knowledge_integration"]) + action = {"value_claims": ["banana"]} + assert tr.score_value(action, [a, b]) == 0.0 +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `python -m pytest tests/test_truth_resolver.py -q -k score_value` +Expected: FAIL — `AttributeError: ... has no attribute 'score_value'`. + +- [ ] **Step 3: Write minimal implementation** + +Add to `scripts/truth_resolver.py`: + +```python +# claim phrase -> (value dimension, polarity) +CLAIM_MAP = { + "no new deps": ("knowledge_integration", +1), + "reuses": ("knowledge_integration", +1), + "dry": ("knowledge_integration", +1), + "isomorphic": ("knowledge_integration", +1), + "low drift": ("knowledge_integration", +1), + "new dependency": ("knowledge_integration", -1), + "duplicate": ("knowledge_integration", -1), + "drift": ("knowledge_integration", -1), + "fork-specific": ("knowledge_integration", -1), + "o(1)": ("performance", +1), + "o(n^2)": ("performance", -1), + "tested": ("correctness", +1), + "untested": ("correctness", -1), + "secure": ("security", +1), + "insecure": ("security", -1), +} + + +def score_value(action: dict, vectors: list[AlignmentVector], + weights: dict | None = None) -> float: + """Net value: each known claim credits its dimension (weighted), but only + if every actor shares that value dimension. Unknown claims are ignored.""" + weights = weights or DEFAULT_WEIGHTS + shared = set.intersection(*[v.value for v in vectors]) if vectors else set() + score = 0.0 + for claim in action.get("value_claims", []): + key = claim.strip().lower() + if key in CLAIM_MAP: + dim, polarity = CLAIM_MAP[key] + if dim in shared: + score += polarity * weights.get(dim, 1.0) + return score +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `python -m pytest tests/test_truth_resolver.py -q` +Expected: PASS (14 tests total). + +- [ ] **Step 5: Commit** + +```bash +git add scripts/truth_resolver.py tests/test_truth_resolver.py +git commit -m "feat(truth-resolver): add shared-value-gated value scoring" +``` + +--- + +## Task 5: `resolve` + `_dissent` (the Verdict) + +**Files:** +- Modify: `scripts/truth_resolver.py` +- Test: `tests/test_truth_resolver.py` + +- [ ] **Step 1: Write the failing test** + +Append: + +```python +def test_resolve_passes_when_aligned_and_value_positive(): + actors = [ + {"role": "user", "intent": "reduce drift", "context": "studio", "values": ["knowledge_integration"]}, + {"role": "agent", "intent": "reduce drift", "context": "studio", "values": ["knowledge_integration"]}, + ] + action = {"summary": "reuse store", "value_claims": ["no new deps"]} + v = tr.resolve(action, actors) + assert v.passed is True + assert v.alignment == 1.0 + assert v.value == 1.0 + assert v.dissent == [] + assert "alignment=1.00" in v.rationale + + +def test_resolve_fails_and_names_dissent_when_misaligned(): + actors = [ + {"role": "user", "intent": "reduce drift", "context": "studio", "values": ["knowledge_integration"]}, + {"role": "subagent", "intent": "rewrite everything", "context": "backend", "values": ["knowledge_integration"]}, + ] + action = {"summary": "x", "value_claims": ["no new deps"]} + v = tr.resolve(action, actors) + assert v.passed is False + # intent and context are disjoint -> both below threshold -> dissent names role:dim + assert "user:intent" in v.dissent or "subagent:intent" in v.dissent + assert any(d.endswith(":context") for d in v.dissent) + + +def test_resolve_fails_when_value_not_positive_even_if_aligned(): + actors = [ + {"role": "user", "intent": "same", "context": "same", "values": ["knowledge_integration"]}, + {"role": "agent", "intent": "same", "context": "same", "values": ["knowledge_integration"]}, + ] + action = {"summary": "x", "value_claims": ["new dependency"]} # -1 value + v = tr.resolve(action, actors) + assert v.alignment == 1.0 + assert v.value == -1.0 + assert v.passed is False + + +def test_resolve_threshold_is_overridable(): + actors = [ + {"role": "user", "intent": "x y", "context": "c", "values": ["knowledge_integration"]}, + {"role": "agent", "intent": "y z", "context": "c", "values": ["knowledge_integration"]}, + ] + action = {"value_claims": ["no new deps"]} + # intent overlap 1/3, context 1.0, value 1.0 -> aggregate ~0.778 + high = tr.resolve(action, actors, threshold=0.9) + assert high.passed is False + low = tr.resolve(action, actors, threshold=0.5) + assert low.passed is True +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `python -m pytest tests/test_truth_resolver.py -q -k resolve` +Expected: FAIL — `AttributeError: ... has no attribute 'resolve'`. + +- [ ] **Step 3: Write minimal implementation** + +Add to `scripts/truth_resolver.py`: + +```python +def _dissent(vectors: list[AlignmentVector], dim_scores: dict, threshold: float) -> list[str]: + """For each below-threshold dimension, name the actors holding tokens that + are not shared by all actors (i.e. the contributors to the misalignment).""" + out: list[str] = [] + for dim in ("intent", "context", "value"): + if dim_scores[dim] >= threshold: + continue + sets = [getattr(v, dim) for v in vectors] + shared = set.intersection(*sets) if sets else set() + for v in vectors: + if getattr(v, dim) - shared: + out.append(f"{v.role}:{dim}") + return out + + +def resolve(action: dict, raw_actors: list[dict], distiller: Distiller | None = None, + threshold: float = ALIGN_THRESHOLD, weights: dict | None = None) -> Verdict: + """Distill actors, score alignment + value, and gate.""" + distiller = distiller or DeterministicDistiller() + vectors = [distiller.distill(a) for a in raw_actors] + ov = overlap(vectors) + alignment = ov["aggregate"] + value = score_value(action, vectors, weights) + passed = alignment >= threshold and value > 0.0 + rationale = ( + f"alignment={alignment:.2f} (>= {threshold}: {alignment >= threshold}); " + f"value={value:.2f} (> 0: {value > 0.0})" + ) + return Verdict( + passed=passed, + alignment=alignment, + value=value, + rationale=rationale, + dissent=_dissent(vectors, ov["dimensions"], threshold), + ) +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `python -m pytest tests/test_truth_resolver.py -q` +Expected: PASS (18 tests total). + +- [ ] **Step 5: Commit** + +```bash +git add scripts/truth_resolver.py tests/test_truth_resolver.py +git commit -m "feat(truth-resolver): add resolve() + dissent reporting" +``` + +--- + +## Task 6: CLI `main()` + interface-satisfiable seam test + fixtures + +**Files:** +- Modify: `scripts/truth_resolver.py` +- Create: `tests/fixtures/action_pass.json`, `tests/fixtures/action_fail.json` +- Test: `tests/test_truth_resolver.py` + +- [ ] **Step 1: Create the fixtures** + +Create `tests/fixtures/action_pass.json`: + +```json +{ + "action": { "summary": "Reuse the existing store", "value_claims": ["no new deps", "reuses"] }, + "actors": [ + { "role": "user", "intent": "reduce drift", "context": "studio", "values": ["knowledge_integration"] }, + { "role": "agent", "intent": "reduce drift", "context": "studio", "values": ["knowledge_integration"] } + ] +} +``` + +Create `tests/fixtures/action_fail.json`: + +```json +{ + "action": { "summary": "Add a new framework", "value_claims": ["new dependency"] }, + "actors": [ + { "role": "user", "intent": "stay minimal", "context": "studio", "values": ["knowledge_integration"] }, + { "role": "subagent", "intent": "introduce library", "context": "backend", "values": ["knowledge_integration"] } + ] +} +``` + +- [ ] **Step 2: Write the failing test** + +Append to `tests/test_truth_resolver.py`: + +```python +FIXTURES = os.path.join(os.path.dirname(__file__), "fixtures") + + +def test_llm_distiller_interface_is_satisfiable_without_implementing_it(): + # The seam is real: a trivial conforming stub passes the Protocol check, + # and the prototype does NOT ship an LLMDistiller class. + class _StubLLMDistiller: + def distill(self, actor): + return tr.AlignmentVector(role=actor.get("role", "x"), + intent=set(), context=set(), value=set()) + assert isinstance(_StubLLMDistiller(), tr.Distiller) + assert not hasattr(tr, "LLMDistiller") + + +def test_main_returns_zero_and_prints_passed_on_pass_fixture(capsys): + rc = tr.main([os.path.join(FIXTURES, "action_pass.json")]) + out = json.loads(capsys.readouterr().out) + assert rc == 0 + assert out["passed"] is True + assert out["value"] == 2.0 + assert out["dissent"] == [] + + +def test_main_returns_one_on_fail_fixture(capsys): + rc = tr.main([os.path.join(FIXTURES, "action_fail.json")]) + out = json.loads(capsys.readouterr().out) + assert rc == 1 + assert out["passed"] is False + + +def test_main_usage_when_no_args(capsys): + rc = tr.main([]) + assert rc == 2 + assert "usage" in capsys.readouterr().out.lower() + + +def test_resolve_runs_with_subprocess_cli(tmp_path): + # End-to-end: invoke the module as a script against the pass fixture. + script = os.path.join(os.path.dirname(__file__), "..", "scripts", "truth_resolver.py") + fixture = os.path.join(FIXTURES, "action_pass.json") + result = subprocess.run([sys.executable, script, fixture], capture_output=True, text=True) + assert result.returncode == 0 + assert '"passed": true' in result.stdout +``` + +- [ ] **Step 3: Run test to verify it fails** + +Run: `python -m pytest tests/test_truth_resolver.py -q -k main or cli or interface` +Expected: FAIL — `AttributeError: ... has no attribute 'main'`. + +- [ ] **Step 4: Write minimal implementation** + +Add to `scripts/truth_resolver.py` (add `import json` and `import sys` to the top imports): + +```python +import json +import sys +``` + +```python +def main(argv: list[str] | None = None) -> int: + """CLI: read an action+actors JSON file, print the Verdict as JSON. + + Returns 0 if the verdict passes, 1 if it fails, 2 on usage error. + """ + argv = list(sys.argv[1:] if argv is None else argv) + if not argv: + print("usage: truth_resolver.py ") + return 2 + with open(argv[0], encoding="utf-8") as f: + doc = json.load(f) + verdict = resolve(doc["action"], doc["actors"]) + print(json.dumps({ + "passed": verdict.passed, + "alignment": round(verdict.alignment, 4), + "value": verdict.value, + "rationale": verdict.rationale, + "dissent": verdict.dissent, + }, indent=2)) + return 0 if verdict.passed else 1 + + +if __name__ == "__main__": + sys.exit(main()) +``` + +- [ ] **Step 5: Run test to verify it passes** + +Run: `python -m pytest tests/test_truth_resolver.py -q` +Expected: PASS (23 tests total). + +- [ ] **Step 6: Final full-suite run** + +Run: `python -m pytest tests/test_truth_resolver.py -q` +Expected: all green. Also sanity-run the CLI by hand: +`python scripts/truth_resolver.py tests/fixtures/action_pass.json` → prints `"passed": true`, exit 0. +`python scripts/truth_resolver.py tests/fixtures/action_fail.json` → prints `"passed": false`, exit 1. + +- [ ] **Step 7: Commit** + +```bash +git add scripts/truth_resolver.py tests/test_truth_resolver.py tests/fixtures/action_pass.json tests/fixtures/action_fail.json +git commit -m "feat(truth-resolver): add CLI main() + seam interface test + fixtures" +``` + +--- + +## Definition of Done + +- `scripts/truth_resolver.py` implements the full pipeline (`AlignmentVector`, `Verdict`, + `Distiller` protocol, `DeterministicDistiller`, `overlap`, `score_value`, `resolve`, `main`). +- `python -m pytest tests/test_truth_resolver.py -q` is green (23 tests), no API key, no network. +- CLI runs on both fixtures (pass → exit 0, fail → exit 1). +- `LLMDistiller` is NOT implemented; the seam is proven satisfiable by a stub (Task 6 test). +- No existing file modified; nothing wired into the evolution loop; R3F work untouched. + +--- + +## Self-Review + +**1. Spec coverage:** §1 purpose → standalone module + CLI (Task 6); §2 pipeline → Tasks 2–5; the `Distiller` seam → Task 1 (protocol) + Task 2 (deterministic) + Task 6 (interface-satisfiable, no `LLMDistiller`); §3 `Verdict` incl. `dissent` → Task 5; §4 `knowledge_integration` value dimension → Task 4 `CLAIM_MAP`; §5 JSON input → Task 6 fixtures + `main`; §6 single file → all tasks; §7 deterministic TDD → every task is red/green; §9 DoD mirrored above; §10 risks (heuristic value, deterministic) honored (pure functions, documented claim map). All spec sections map to a task. ✅ + +**2. Placeholder scan:** No TBD/TODO; every step has runnable code and exact commands. ✅ + +**3. Type/name consistency:** `AlignmentVector{role,intent,context,value}`, `Verdict{passed,alignment,value,rationale,dissent}`, `overlap()->{"dimensions","aggregate"}`, `score_value(action,vectors,weights)`, `resolve(action,raw_actors,distiller,threshold,weights)`, `_tokens`, `_jaccard_all`, `_dissent`, `CLAIM_MAP`, `ALIGN_THRESHOLD`, `VALUE_DIMENSIONS`, `DEFAULT_WEIGHTS`, `main(argv)` — all referenced consistently across tasks and tests. The test file uses `tr.` throughout so partial implementations fail with `AttributeError` (clean RED). ✅ From e1917673fd0836516f163eaba22584dc9cd1ba62 Mon Sep 17 00:00:00 2001 From: ZuluYokohama Date: Sat, 6 Jun 2026 00:57:10 -0500 Subject: [PATCH 03/10] feat(truth-resolver): scaffold dataclasses + Distiller protocol --- scripts/truth_resolver.py | 40 ++++++++++++++++++++++++++++++++++++ tests/test_truth_resolver.py | 28 +++++++++++++++++++++++++ 2 files changed, 68 insertions(+) create mode 100644 scripts/truth_resolver.py create mode 100644 tests/test_truth_resolver.py diff --git a/scripts/truth_resolver.py b/scripts/truth_resolver.py new file mode 100644 index 0000000..90f3c06 --- /dev/null +++ b/scripts/truth_resolver.py @@ -0,0 +1,40 @@ +"""Truth-Resolver prototype gate. + +Resolves whether a proposed action is aligned across the involved actors' +intent / context / value, and net-positive on value. Deterministic pipeline +with the single semantic step (distillation) isolated behind the `Distiller` +interface so an LLM-backed distiller can drop in later without a rewrite. +""" +from __future__ import annotations + +from dataclasses import dataclass +from typing import Protocol, runtime_checkable + +ALIGN_THRESHOLD = 0.5 +VALUE_DIMENSIONS = ("correctness", "performance", "security", "knowledge_integration") +DEFAULT_WEIGHTS = {d: 1.0 for d in VALUE_DIMENSIONS} + + +@dataclass +class AlignmentVector: + """One actor's distilled stance, as token/tag sets per dimension.""" + role: str + intent: set[str] + context: set[str] + value: set[str] + + +@dataclass +class Verdict: + """The resolver's decision for an action.""" + passed: bool + alignment: float + value: float + rationale: str + dissent: list[str] + + +@runtime_checkable +class Distiller(Protocol): + """The pluggable seam: turn a raw actor dict into an AlignmentVector.""" + def distill(self, actor: dict) -> AlignmentVector: ... diff --git a/tests/test_truth_resolver.py b/tests/test_truth_resolver.py new file mode 100644 index 0000000..843053b --- /dev/null +++ b/tests/test_truth_resolver.py @@ -0,0 +1,28 @@ +import os +import sys +import json +import subprocess + +sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..', 'scripts'))) +import truth_resolver as tr # noqa: E402 + + +def test_constants_and_dataclasses_exist(): + assert tr.ALIGN_THRESHOLD == 0.5 + assert tr.VALUE_DIMENSIONS == ("correctness", "performance", "security", "knowledge_integration") + + v = tr.AlignmentVector(role="user", intent={"a"}, context={"b"}, value={"correctness"}) + assert v.role == "user" + assert v.intent == {"a"} and v.context == {"b"} and v.value == {"correctness"} + + verdict = tr.Verdict(passed=True, alignment=1.0, value=2.0, rationale="ok", dissent=[]) + assert verdict.passed is True and verdict.alignment == 1.0 + assert verdict.value == 2.0 and verdict.rationale == "ok" and verdict.dissent == [] + + +def test_distiller_is_runtime_checkable_protocol(): + class StubDistiller: + def distill(self, actor): + return tr.AlignmentVector(role="x", intent=set(), context=set(), value=set()) + assert isinstance(StubDistiller(), tr.Distiller) + assert not isinstance(object(), tr.Distiller) From 917006d0490c6d6153fed05e70a89805d3b5dee6 Mon Sep 17 00:00:00 2001 From: ZuluYokohama Date: Sat, 6 Jun 2026 00:57:45 -0500 Subject: [PATCH 04/10] feat(truth-resolver): add DeterministicDistiller + tokenizer --- scripts/truth_resolver.py | 17 +++++++++++++++++ tests/test_truth_resolver.py | 27 +++++++++++++++++++++++++++ 2 files changed, 44 insertions(+) diff --git a/scripts/truth_resolver.py b/scripts/truth_resolver.py index 90f3c06..828e9e3 100644 --- a/scripts/truth_resolver.py +++ b/scripts/truth_resolver.py @@ -7,6 +7,7 @@ """ from __future__ import annotations +import re from dataclasses import dataclass from typing import Protocol, runtime_checkable @@ -38,3 +39,19 @@ class Verdict: class Distiller(Protocol): """The pluggable seam: turn a raw actor dict into an AlignmentVector.""" def distill(self, actor: dict) -> AlignmentVector: ... + + +def _tokens(text: str) -> set[str]: + """Lowercase, split on non-alphanumerics, drop empties.""" + return {t for t in re.split(r"[^a-z0-9]+", text.lower()) if t} + + +class DeterministicDistiller: + """Default distiller: normalize free text into token/tag sets. No LLM.""" + def distill(self, actor: dict) -> AlignmentVector: + return AlignmentVector( + role=actor.get("role", "unknown"), + intent=_tokens(actor.get("intent", "")), + context=_tokens(actor.get("context", "")), + value={v.strip().lower() for v in actor.get("values", [])}, + ) diff --git a/tests/test_truth_resolver.py b/tests/test_truth_resolver.py index 843053b..e955416 100644 --- a/tests/test_truth_resolver.py +++ b/tests/test_truth_resolver.py @@ -26,3 +26,30 @@ def distill(self, actor): return tr.AlignmentVector(role="x", intent=set(), context=set(), value=set()) assert isinstance(StubDistiller(), tr.Distiller) assert not isinstance(object(), tr.Distiller) + + +def test_tokens_splits_lowercases_and_drops_empties(): + assert tr._tokens("O(1) memory, Zero-Drift!") == {"o", "1", "memory", "zero", "drift"} + assert tr._tokens("") == set() + + +def test_deterministic_distiller_builds_vector(): + d = tr.DeterministicDistiller() + actor = { + "role": "user", + "intent": "Reduce drift to O(1)", + "context": "studio build", + "values": ["Correctness", "Knowledge_Integration"], + } + v = d.distill(actor) + assert isinstance(v, tr.AlignmentVector) + assert v.role == "user" + assert v.intent == {"reduce", "drift", "to", "o", "1"} + assert v.context == {"studio", "build"} + assert v.value == {"correctness", "knowledge_integration"} # lowercased + + +def test_deterministic_distiller_defaults_for_missing_fields(): + v = tr.DeterministicDistiller().distill({}) + assert v.role == "unknown" + assert v.intent == set() and v.context == set() and v.value == set() From 94d22f76eece02d9ffa2936c23b91cf0f688496b Mon Sep 17 00:00:00 2001 From: ZuluYokohama Date: Sat, 6 Jun 2026 00:58:14 -0500 Subject: [PATCH 05/10] feat(truth-resolver): add per-dimension Jaccard overlap --- scripts/truth_resolver.py | 25 ++++++++++++++++++++++++ tests/test_truth_resolver.py | 37 ++++++++++++++++++++++++++++++++++++ 2 files changed, 62 insertions(+) diff --git a/scripts/truth_resolver.py b/scripts/truth_resolver.py index 828e9e3..3b6a34d 100644 --- a/scripts/truth_resolver.py +++ b/scripts/truth_resolver.py @@ -55,3 +55,28 @@ def distill(self, actor: dict) -> AlignmentVector: context=_tokens(actor.get("context", "")), value={v.strip().lower() for v in actor.get("values", [])}, ) + + +def _jaccard_all(sets: list[set[str]]) -> float: + """Multi-set Jaccard: |intersection of all| / |union of all|. + + Empty union (every actor empty on this dimension) is vacuously aligned -> 1.0. + """ + if not sets: + return 1.0 + union: set[str] = set().union(*sets) + if not union: + return 1.0 + inter = set(sets[0]).intersection(*sets[1:]) + return len(inter) / len(union) + + +def overlap(vectors: list[AlignmentVector]) -> dict: + """Per-dimension and aggregate alignment across actors.""" + dims = { + "intent": _jaccard_all([v.intent for v in vectors]), + "context": _jaccard_all([v.context for v in vectors]), + "value": _jaccard_all([v.value for v in vectors]), + } + aggregate = sum(dims.values()) / len(dims) + return {"dimensions": dims, "aggregate": aggregate} diff --git a/tests/test_truth_resolver.py b/tests/test_truth_resolver.py index e955416..58c9ef2 100644 --- a/tests/test_truth_resolver.py +++ b/tests/test_truth_resolver.py @@ -53,3 +53,40 @@ def test_deterministic_distiller_defaults_for_missing_fields(): v = tr.DeterministicDistiller().distill({}) assert v.role == "unknown" assert v.intent == set() and v.context == set() and v.value == set() + + +def _vec(role, intent, context, value): + return tr.AlignmentVector(role=role, intent=set(intent), context=set(context), value=set(value)) + + +def test_overlap_identical_actors_is_one(): + a = _vec("user", ["x", "y"], ["c"], ["correctness"]) + b = _vec("agent", ["x", "y"], ["c"], ["correctness"]) + ov = tr.overlap([a, b]) + assert ov["dimensions"] == {"intent": 1.0, "context": 1.0, "value": 1.0} + assert ov["aggregate"] == 1.0 + + +def test_overlap_disjoint_actors_is_zero(): + a = _vec("user", ["x"], ["c1"], ["correctness"]) + b = _vec("agent", ["y"], ["c2"], ["performance"]) + ov = tr.overlap([a, b]) + assert ov["dimensions"] == {"intent": 0.0, "context": 0.0, "value": 0.0} + assert ov["aggregate"] == 0.0 + + +def test_overlap_partial_is_exact_jaccard(): + # intent: {x,y} vs {y,z} -> inter {y}=1, union {x,y,z}=3 -> 1/3 + a = _vec("user", ["x", "y"], ["c"], ["correctness"]) + b = _vec("agent", ["y", "z"], ["c"], ["correctness"]) + ov = tr.overlap([a, b]) + assert abs(ov["dimensions"]["intent"] - (1 / 3)) < 1e-9 + assert ov["dimensions"]["context"] == 1.0 + assert ov["dimensions"]["value"] == 1.0 + + +def test_overlap_all_empty_dimension_is_vacuously_one(): + a = _vec("user", [], [], []) + b = _vec("agent", [], [], []) + ov = tr.overlap([a, b]) + assert ov["aggregate"] == 1.0 From e5f5aef7a18cb7b8865b80e4c6b3d0262bf1fcb6 Mon Sep 17 00:00:00 2001 From: ZuluYokohama Date: Sat, 6 Jun 2026 00:58:45 -0500 Subject: [PATCH 06/10] feat(truth-resolver): add shared-value-gated value scoring --- scripts/truth_resolver.py | 36 ++++++++++++++++++++++++++++++++++++ tests/test_truth_resolver.py | 36 ++++++++++++++++++++++++++++++++++++ 2 files changed, 72 insertions(+) diff --git a/scripts/truth_resolver.py b/scripts/truth_resolver.py index 3b6a34d..b54dc03 100644 --- a/scripts/truth_resolver.py +++ b/scripts/truth_resolver.py @@ -80,3 +80,39 @@ def overlap(vectors: list[AlignmentVector]) -> dict: } aggregate = sum(dims.values()) / len(dims) return {"dimensions": dims, "aggregate": aggregate} + + +# claim phrase -> (value dimension, polarity) +CLAIM_MAP = { + "no new deps": ("knowledge_integration", +1), + "reuses": ("knowledge_integration", +1), + "dry": ("knowledge_integration", +1), + "isomorphic": ("knowledge_integration", +1), + "low drift": ("knowledge_integration", +1), + "new dependency": ("knowledge_integration", -1), + "duplicate": ("knowledge_integration", -1), + "drift": ("knowledge_integration", -1), + "fork-specific": ("knowledge_integration", -1), + "o(1)": ("performance", +1), + "o(n^2)": ("performance", -1), + "tested": ("correctness", +1), + "untested": ("correctness", -1), + "secure": ("security", +1), + "insecure": ("security", -1), +} + + +def score_value(action: dict, vectors: list[AlignmentVector], + weights: dict | None = None) -> float: + """Net value: each known claim credits its dimension (weighted), but only + if every actor shares that value dimension. Unknown claims are ignored.""" + weights = weights or DEFAULT_WEIGHTS + shared = set.intersection(*[v.value for v in vectors]) if vectors else set() + score = 0.0 + for claim in action.get("value_claims", []): + key = claim.strip().lower() + if key in CLAIM_MAP: + dim, polarity = CLAIM_MAP[key] + if dim in shared: + score += polarity * weights.get(dim, 1.0) + return score diff --git a/tests/test_truth_resolver.py b/tests/test_truth_resolver.py index 58c9ef2..4465d09 100644 --- a/tests/test_truth_resolver.py +++ b/tests/test_truth_resolver.py @@ -90,3 +90,39 @@ def test_overlap_all_empty_dimension_is_vacuously_one(): b = _vec("agent", [], [], []) ov = tr.overlap([a, b]) assert ov["aggregate"] == 1.0 + + +def test_score_value_credits_positive_claim_when_dimension_shared(): + a = _vec("user", ["i"], ["c"], ["knowledge_integration"]) + b = _vec("agent", ["i"], ["c"], ["knowledge_integration"]) + action = {"value_claims": ["no new deps", "reuses"]} + # both claims map to knowledge_integration (+1 each), dim shared -> +2.0 + assert tr.score_value(action, [a, b]) == 2.0 + + +def test_score_value_no_credit_when_dimension_not_shared(): + a = _vec("user", ["i"], ["c"], ["correctness"]) + b = _vec("agent", ["i"], ["c"], ["performance"]) # no shared value dim + action = {"value_claims": ["no new deps"]} + assert tr.score_value(action, [a, b]) == 0.0 + + +def test_score_value_negative_claims_subtract(): + a = _vec("user", ["i"], ["c"], ["knowledge_integration"]) + b = _vec("agent", ["i"], ["c"], ["knowledge_integration"]) + action = {"value_claims": ["new dependency", "duplicate"]} + assert tr.score_value(action, [a, b]) == -2.0 + + +def test_score_value_maps_claims_to_correct_dimensions(): + a = _vec("user", ["i"], ["c"], ["performance", "correctness"]) + b = _vec("agent", ["i"], ["c"], ["performance", "correctness"]) + action = {"value_claims": ["o(1)", "tested"]} # performance +1, correctness +1 + assert tr.score_value(action, [a, b]) == 2.0 + + +def test_score_value_ignores_unknown_claims(): + a = _vec("user", ["i"], ["c"], ["knowledge_integration"]) + b = _vec("agent", ["i"], ["c"], ["knowledge_integration"]) + action = {"value_claims": ["banana"]} + assert tr.score_value(action, [a, b]) == 0.0 From b8e7e3989e8a12603a0dbd81fbd8c0b05b5a17bf Mon Sep 17 00:00:00 2001 From: ZuluYokohama Date: Sat, 6 Jun 2026 00:59:21 -0500 Subject: [PATCH 07/10] feat(truth-resolver): add resolve() + dissent reporting --- scripts/truth_resolver.py | 37 +++++++++++++++++++++++++ tests/test_truth_resolver.py | 52 ++++++++++++++++++++++++++++++++++++ 2 files changed, 89 insertions(+) diff --git a/scripts/truth_resolver.py b/scripts/truth_resolver.py index b54dc03..aa7f758 100644 --- a/scripts/truth_resolver.py +++ b/scripts/truth_resolver.py @@ -116,3 +116,40 @@ def score_value(action: dict, vectors: list[AlignmentVector], if dim in shared: score += polarity * weights.get(dim, 1.0) return score + + +def _dissent(vectors: list[AlignmentVector], dim_scores: dict, threshold: float) -> list[str]: + """For each below-threshold dimension, name the actors holding tokens that + are not shared by all actors (i.e. the contributors to the misalignment).""" + out: list[str] = [] + for dim in ("intent", "context", "value"): + if dim_scores[dim] >= threshold: + continue + sets = [getattr(v, dim) for v in vectors] + shared = set.intersection(*sets) if sets else set() + for v in vectors: + if getattr(v, dim) - shared: + out.append(f"{v.role}:{dim}") + return out + + +def resolve(action: dict, raw_actors: list[dict], distiller: Distiller | None = None, + threshold: float = ALIGN_THRESHOLD, weights: dict | None = None) -> Verdict: + """Distill actors, score alignment + value, and gate.""" + distiller = distiller or DeterministicDistiller() + vectors = [distiller.distill(a) for a in raw_actors] + ov = overlap(vectors) + alignment = ov["aggregate"] + value = score_value(action, vectors, weights) + passed = alignment >= threshold and value > 0.0 + rationale = ( + f"alignment={alignment:.2f} (>= {threshold}: {alignment >= threshold}); " + f"value={value:.2f} (> 0: {value > 0.0})" + ) + return Verdict( + passed=passed, + alignment=alignment, + value=value, + rationale=rationale, + dissent=_dissent(vectors, ov["dimensions"], threshold), + ) diff --git a/tests/test_truth_resolver.py b/tests/test_truth_resolver.py index 4465d09..f15e538 100644 --- a/tests/test_truth_resolver.py +++ b/tests/test_truth_resolver.py @@ -126,3 +126,55 @@ def test_score_value_ignores_unknown_claims(): b = _vec("agent", ["i"], ["c"], ["knowledge_integration"]) action = {"value_claims": ["banana"]} assert tr.score_value(action, [a, b]) == 0.0 + + +def test_resolve_passes_when_aligned_and_value_positive(): + actors = [ + {"role": "user", "intent": "reduce drift", "context": "studio", "values": ["knowledge_integration"]}, + {"role": "agent", "intent": "reduce drift", "context": "studio", "values": ["knowledge_integration"]}, + ] + action = {"summary": "reuse store", "value_claims": ["no new deps"]} + v = tr.resolve(action, actors) + assert v.passed is True + assert v.alignment == 1.0 + assert v.value == 1.0 + assert v.dissent == [] + assert "alignment=1.00" in v.rationale + + +def test_resolve_fails_and_names_dissent_when_misaligned(): + actors = [ + {"role": "user", "intent": "reduce drift", "context": "studio", "values": ["knowledge_integration"]}, + {"role": "subagent", "intent": "rewrite everything", "context": "backend", "values": ["knowledge_integration"]}, + ] + action = {"summary": "x", "value_claims": ["no new deps"]} + v = tr.resolve(action, actors) + assert v.passed is False + # intent and context are disjoint -> both below threshold -> dissent names role:dim + assert "user:intent" in v.dissent or "subagent:intent" in v.dissent + assert any(d.endswith(":context") for d in v.dissent) + + +def test_resolve_fails_when_value_not_positive_even_if_aligned(): + actors = [ + {"role": "user", "intent": "same", "context": "same", "values": ["knowledge_integration"]}, + {"role": "agent", "intent": "same", "context": "same", "values": ["knowledge_integration"]}, + ] + action = {"summary": "x", "value_claims": ["new dependency"]} # -1 value + v = tr.resolve(action, actors) + assert v.alignment == 1.0 + assert v.value == -1.0 + assert v.passed is False + + +def test_resolve_threshold_is_overridable(): + actors = [ + {"role": "user", "intent": "x y", "context": "c", "values": ["knowledge_integration"]}, + {"role": "agent", "intent": "y z", "context": "c", "values": ["knowledge_integration"]}, + ] + action = {"value_claims": ["no new deps"]} + # intent overlap 1/3, context 1.0, value 1.0 -> aggregate ~0.778 + high = tr.resolve(action, actors, threshold=0.9) + assert high.passed is False + low = tr.resolve(action, actors, threshold=0.5) + assert low.passed is True From 7fd54e18d4605c433fca03319ec11a4455c93ecb Mon Sep 17 00:00:00 2001 From: ZuluYokohama Date: Sat, 6 Jun 2026 01:00:16 -0500 Subject: [PATCH 08/10] feat(truth-resolver): add CLI main() + seam interface test + fixtures --- scripts/truth_resolver.py | 28 ++++++++++++++++++++ tests/fixtures/action_fail.json | 7 +++++ tests/fixtures/action_pass.json | 7 +++++ tests/test_truth_resolver.py | 45 +++++++++++++++++++++++++++++++++ 4 files changed, 87 insertions(+) create mode 100644 tests/fixtures/action_fail.json create mode 100644 tests/fixtures/action_pass.json diff --git a/scripts/truth_resolver.py b/scripts/truth_resolver.py index aa7f758..6403dd8 100644 --- a/scripts/truth_resolver.py +++ b/scripts/truth_resolver.py @@ -7,7 +7,9 @@ """ from __future__ import annotations +import json import re +import sys from dataclasses import dataclass from typing import Protocol, runtime_checkable @@ -153,3 +155,29 @@ def resolve(action: dict, raw_actors: list[dict], distiller: Distiller | None = rationale=rationale, dissent=_dissent(vectors, ov["dimensions"], threshold), ) + + +def main(argv: list[str] | None = None) -> int: + """CLI: read an action+actors JSON file, print the Verdict as JSON. + + Returns 0 if the verdict passes, 1 if it fails, 2 on usage error. + """ + argv = list(sys.argv[1:] if argv is None else argv) + if not argv: + print("usage: truth_resolver.py ") + return 2 + with open(argv[0], encoding="utf-8") as f: + doc = json.load(f) + verdict = resolve(doc["action"], doc["actors"]) + print(json.dumps({ + "passed": verdict.passed, + "alignment": round(verdict.alignment, 4), + "value": verdict.value, + "rationale": verdict.rationale, + "dissent": verdict.dissent, + }, indent=2)) + return 0 if verdict.passed else 1 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/tests/fixtures/action_fail.json b/tests/fixtures/action_fail.json new file mode 100644 index 0000000..0e4703a --- /dev/null +++ b/tests/fixtures/action_fail.json @@ -0,0 +1,7 @@ +{ + "action": { "summary": "Add a new framework", "value_claims": ["new dependency"] }, + "actors": [ + { "role": "user", "intent": "stay minimal", "context": "studio", "values": ["knowledge_integration"] }, + { "role": "subagent", "intent": "introduce library", "context": "backend", "values": ["knowledge_integration"] } + ] +} diff --git a/tests/fixtures/action_pass.json b/tests/fixtures/action_pass.json new file mode 100644 index 0000000..2d21333 --- /dev/null +++ b/tests/fixtures/action_pass.json @@ -0,0 +1,7 @@ +{ + "action": { "summary": "Reuse the existing store", "value_claims": ["no new deps", "reuses"] }, + "actors": [ + { "role": "user", "intent": "reduce drift", "context": "studio", "values": ["knowledge_integration"] }, + { "role": "agent", "intent": "reduce drift", "context": "studio", "values": ["knowledge_integration"] } + ] +} diff --git a/tests/test_truth_resolver.py b/tests/test_truth_resolver.py index f15e538..0f84de2 100644 --- a/tests/test_truth_resolver.py +++ b/tests/test_truth_resolver.py @@ -178,3 +178,48 @@ def test_resolve_threshold_is_overridable(): assert high.passed is False low = tr.resolve(action, actors, threshold=0.5) assert low.passed is True + + +FIXTURES = os.path.join(os.path.dirname(__file__), "fixtures") + + +def test_llm_distiller_interface_is_satisfiable_without_implementing_it(): + # The seam is real: a trivial conforming stub passes the Protocol check, + # and the prototype does NOT ship an LLMDistiller class. + class _StubLLMDistiller: + def distill(self, actor): + return tr.AlignmentVector(role=actor.get("role", "x"), + intent=set(), context=set(), value=set()) + assert isinstance(_StubLLMDistiller(), tr.Distiller) + assert not hasattr(tr, "LLMDistiller") + + +def test_main_returns_zero_and_prints_passed_on_pass_fixture(capsys): + rc = tr.main([os.path.join(FIXTURES, "action_pass.json")]) + out = json.loads(capsys.readouterr().out) + assert rc == 0 + assert out["passed"] is True + assert out["value"] == 2.0 + assert out["dissent"] == [] + + +def test_main_returns_one_on_fail_fixture(capsys): + rc = tr.main([os.path.join(FIXTURES, "action_fail.json")]) + out = json.loads(capsys.readouterr().out) + assert rc == 1 + assert out["passed"] is False + + +def test_main_usage_when_no_args(capsys): + rc = tr.main([]) + assert rc == 2 + assert "usage" in capsys.readouterr().out.lower() + + +def test_resolve_runs_with_subprocess_cli(tmp_path): + # End-to-end: invoke the module as a script against the pass fixture. + script = os.path.join(os.path.dirname(__file__), "..", "scripts", "truth_resolver.py") + fixture = os.path.join(FIXTURES, "action_pass.json") + result = subprocess.run([sys.executable, script, fixture], capture_output=True, text=True) + assert result.returncode == 0 + assert '"passed": true' in result.stdout From f38d51901dd964ba1f06bbc5519f946b0210c8df Mon Sep 17 00:00:00 2001 From: ZuluYokohama Date: Sat, 6 Jun 2026 01:04:45 -0500 Subject: [PATCH 09/10] fix(truth-resolver): harden CLI on bad input; test weights override - main() returns 2 (not a traceback) on missing file / bad JSON / missing keys - add tests for the error paths and for the weights override + partial fallback --- scripts/truth_resolver.py | 10 +++++++--- tests/test_truth_resolver.py | 23 +++++++++++++++++++++++ 2 files changed, 30 insertions(+), 3 deletions(-) diff --git a/scripts/truth_resolver.py b/scripts/truth_resolver.py index 6403dd8..0dc0477 100644 --- a/scripts/truth_resolver.py +++ b/scripts/truth_resolver.py @@ -166,9 +166,13 @@ def main(argv: list[str] | None = None) -> int: if not argv: print("usage: truth_resolver.py ") return 2 - with open(argv[0], encoding="utf-8") as f: - doc = json.load(f) - verdict = resolve(doc["action"], doc["actors"]) + try: + with open(argv[0], encoding="utf-8") as f: + doc = json.load(f) + verdict = resolve(doc["action"], doc["actors"]) + except (OSError, json.JSONDecodeError, KeyError) as exc: + print(f"error: {exc}") + return 2 print(json.dumps({ "passed": verdict.passed, "alignment": round(verdict.alignment, 4), diff --git a/tests/test_truth_resolver.py b/tests/test_truth_resolver.py index 0f84de2..dab040a 100644 --- a/tests/test_truth_resolver.py +++ b/tests/test_truth_resolver.py @@ -223,3 +223,26 @@ def test_resolve_runs_with_subprocess_cli(tmp_path): result = subprocess.run([sys.executable, script, fixture], capture_output=True, text=True) assert result.returncode == 0 assert '"passed": true' in result.stdout + + +def test_main_returns_2_on_missing_file(capsys): + rc = tr.main([os.path.join(FIXTURES, "does_not_exist.json")]) + assert rc == 2 + assert "error" in capsys.readouterr().out.lower() + + +def test_main_returns_2_on_missing_action_key(tmp_path, capsys): + bad = tmp_path / "bad.json" + bad.write_text('{"actors": []}', encoding="utf-8") # missing "action" key + rc = tr.main([str(bad)]) + assert rc == 2 + assert "error" in capsys.readouterr().out.lower() + + +def test_score_value_respects_weights_override_and_partial_fallback(): + a = _vec("user", ["i"], ["c"], ["performance", "correctness"]) + b = _vec("agent", ["i"], ["c"], ["performance", "correctness"]) + action = {"value_claims": ["o(1)", "tested"]} # performance +1, correctness +1 + # Override performance weight to 3.0; correctness is absent -> falls back to 1.0. + score = tr.score_value(action, [a, b], weights={"performance": 3.0}) + assert score == 4.0 From 9381534e10fe99eef1c95e0c66c0e3012d447e68 Mon Sep 17 00:00:00 2001 From: ZuluYokohama Date: Sat, 6 Jun 2026 01:15:12 -0500 Subject: [PATCH 10/10] chore(truth-resolver): address CodeRabbit nits - spec: add 'text' language to the pipeline code fence (markdownlint MD040) - truth_resolver.py: DEFAULT_WEIGHTS via dict.fromkeys (equivalent, clearer) --- .../specs/2026-06-06-truth-resolver-prototype-design.md | 2 +- scripts/truth_resolver.py | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/superpowers/specs/2026-06-06-truth-resolver-prototype-design.md b/docs/superpowers/specs/2026-06-06-truth-resolver-prototype-design.md index 18717ca..0ca7281 100644 --- a/docs/superpowers/specs/2026-06-06-truth-resolver-prototype-design.md +++ b/docs/superpowers/specs/2026-06-06-truth-resolver-prototype-design.md @@ -26,7 +26,7 @@ or touch the in-flight R3F work. Each actor contributes an `AlignmentVector { intent, context, value }`. The pipeline isolates the semantic step to `Distiller.distill`: -``` +```text Distiller.distill(raw_actor) -> AlignmentVector # ← THE SEAM (deterministic now, LLM later) overlap(vectors) -> per-dimension alignment 0..1 # deterministic (Jaccard) score_value(action, vectors) -> net value (float) # deterministic (weighted delta) diff --git a/scripts/truth_resolver.py b/scripts/truth_resolver.py index 0dc0477..d98fb27 100644 --- a/scripts/truth_resolver.py +++ b/scripts/truth_resolver.py @@ -15,7 +15,7 @@ ALIGN_THRESHOLD = 0.5 VALUE_DIMENSIONS = ("correctness", "performance", "security", "knowledge_integration") -DEFAULT_WEIGHTS = {d: 1.0 for d in VALUE_DIMENSIONS} +DEFAULT_WEIGHTS = dict.fromkeys(VALUE_DIMENSIONS, 1.0) @dataclass