Merge pull request #143 from engineeredsoftware/v39/gate-8-operational-telemetry-repair-readback

V39 Gate 8: Operational Telemetry Repair Readback

Author: geraldarthurdavis

.bitcode/v39-operational-telemetry-repair-readback.json

@@ -202,6 +202,9 @@ jobs:
               if [ -f scripts/check-v39-gate7-settlement-rights-delivery.mjs ]; then
                 node scripts/check-v39-gate7-settlement-rights-delivery.mjs --skip-branch-check --skip-package-tests
               fi
+              if [ -f scripts/check-v39-gate8-operational-telemetry-repair-readback.mjs ]; then
+                node scripts/check-v39-gate8-operational-telemetry-repair-readback.mjs --skip-branch-check --skip-package-tests --skip-uapi-tests
+              fi
             fi
           else
             echo "Unexpected BITCODE_SPEC.txt pointer: $POINTER" >&2

.github/workflows/bitcode-canon-quality.yml

@@ -331,6 +331,9 @@ jobs:
               if [ -f scripts/check-v39-gate7-settlement-rights-delivery.mjs ]; then
                 node scripts/check-v39-gate7-settlement-rights-delivery.mjs --skip-branch-check --skip-package-tests
               fi
+              if [ -f scripts/check-v39-gate8-operational-telemetry-repair-readback.mjs ]; then
+                node scripts/check-v39-gate8-operational-telemetry-repair-readback.mjs --skip-branch-check --skip-package-tests --skip-uapi-tests
+              fi
             fi
           else
             echo "Unexpected BITCODE_SPEC.txt pointer: $POINTER" >&2
@@ -409,6 +412,7 @@ jobs:
           pnpm --filter @bitcode/pipeline-asset-pack exec jest --config jest.config.cjs --runTestsByPath src/__tests__/asset-pack-disclosure.test.ts src/__tests__/postprocess.test.ts src/__tests__/read-need.test.ts src/__tests__/read-need-review-resynthesis.test.ts --runInBand --forceExit
           pnpm --filter @bitcode/pipeline-asset-pack exec jest --config jest.config.cjs --runTestsByPath src/__tests__/asset-pack-preview-boundary.test.ts --runInBand --forceExit
           pnpm --filter @bitcode/pipeline-asset-pack exec jest --config jest.config.cjs --runTestsByPath src/__tests__/asset-pack-settlement-rights-delivery.test.ts --runInBand --forceExit
+          pnpm --filter @bitcode/pipeline-asset-pack exec jest --config jest.config.cjs --runTestsByPath src/__tests__/reading-operational-telemetry-repair-readback.test.ts --runInBand --forceExit
           pnpm --filter @bitcode/api exec jest --config jest.config.cjs --runTestsByPath src/conversations/__tests__/stream-events.test.ts src/conversations/__tests__/privacy.test.ts src/conversations/__tests__/telemetry.test.ts --runInBand --forceExit
           pnpm --dir packages/protocol exec node --test --test-force-exit test/conversation-telemetry-proof-hooks.test.js
           pnpm --dir packages/protocol exec node --test --test-force-exit test/conversation-rehearsal.test.js
@@ -490,6 +494,7 @@ jobs:
             tests/terminalTransactionDetailSnapshot.test.ts \
             tests/terminalUxBrowserProof.test.tsx \
             tests/bitcodeBrowserAccessibilityResponsiveProof.test.ts \
+            tests/readingOperationalTelemetryPipelineLog.test.tsx \
             tests/pipelineExecutionLogHeader.test.tsx \
             --runInBand
 

.github/workflows/bitcode-gate-quality.yml

@@ -187,6 +187,32 @@ responses. The deterministic source-safe proof artifact is
 Gate 8 makes the Reading product observable and repairable.
 It must stream source-safe phase, agent, step, Failsafe, ThricifiedGeneration, tool, storage, ledger, wallet, delivery, and UI events into the rich execution log and operator readback surfaces with proof roots and runbook hooks.
 
+The package-owned closure type is `ReadingOperationalTelemetryRepairReadback`
+in
+`packages/pipelines/asset-pack/src/reading-operational-telemetry-repair-readback.ts`.
+It projects `ReadNeedComprehensionSynthesis` and
+`ReadFitsFindingSynthesis` pipeline contracts into
+`ReadingOperationalTelemetryEvent` rows for phase, PTRR agent, PTRR step,
+`FailsafeGenerationSequence`, `ThricifiedGeneration`, `ToolExecution`,
+storage, ledger, wallet, delivery, UI, and repair posture. Each event carries
+`ReadingOperationalExecutionState` with event id, proof root,
+prompt-template identity, output schema/return type, redaction posture,
+prompt disclosure posture, result disclosure posture, and fail-closed state
+where applicable.
+
+The operator readback persists `ReadingOperationalOperatorReadback`,
+source-safe stream events, repair runbook hooks, telemetry roots, repair
+roots, and readback roots under `reading/operational`. The shared rich
+execution log accepts direct operational telemetry payloads and renders
+Reading pipeline, phase, PTRR agent, PTRR step, Failsafe, generation, tool,
+schema, event, proof, disclosure, and fail-closed metadata in collapsed rows
+with expandable details. No operational artifact may serialize protected
+source, raw protected prompts, raw interpolated prompts, raw provider
+responses, unpaid AssetPack source, wallet private material, private
+settlement payloads, or credentials. The deterministic source-safe proof
+artifact is `.bitcode/v39-operational-telemetry-repair-readback.json`,
+checked by `pnpm run check:v39-gate8`.
+
 ### Gate 9: Interface And Conversation Product Parity
 
 Gate 9 brings Conversation, MCP/API, ChatGPT App, and package-facing interfaces into the same Reading contract without creating parallel authority.

BITCODE_SPEC_V39.md

@@ -199,7 +199,12 @@ Closure acceptance:
 
 Closure implementation:
 
-- Pending Gate 8 work must define telemetry/repair artifacts and route or component tests.
+- Gate 8 package source is `packages/pipelines/asset-pack/src/reading-operational-telemetry-repair-readback.ts`.
+- `ReadingOperationalTelemetryRepairReadback` emits `ReadingOperationalTelemetryEvent`, `ReadingOperationalOperatorReadback`, `ReadingOperationalTelemetryStorageRecord`, and `ReadingOperationalRepairRunbookHook` projections under `reading/operational`.
+- Event kinds are phase, PTRR agent, PTRR step, Failsafe, ThricifiedGeneration, ToolExecution, storage, ledger, wallet, delivery, UI, and repair; each event carries proof roots plus prompt/result disclosure posture.
+- The rich execution log and header accept direct Reading operational telemetry payloads and render pipeline, phase, agent, step, Failsafe, generation, tool, schema, event, proof, redaction, prompt disclosure, result disclosure, and fail-closed posture.
+- Focused package tests cover complete settled stream coverage, BTC-finality repair, and persistence; focused UI tests cover direct Reading telemetry rendering.
+- The generated proof artifact is `.bitcode/v39-operational-telemetry-repair-readback.json`; `pnpm run check:v39-gate8` validates artifact freshness, protocol tests, package tests, UI tests, docs, source safety, and workflow wiring.
 
 ### Gate 9: Interface And Conversation Product Parity
 

BITCODE_SPEC_V39_DELTA.md

@@ -254,7 +254,25 @@ credential value.
 
 ## Gate 8 implementation notes
 
-Gate 8 closes operational telemetry, repair, and operator readback for Reading. Every pipeline, storage, ledger, wallet, delivery, and UI event must be source-safe, proof-rooted, and tied to repair/runbook posture.
+Gate 8 closes operational telemetry, repair, and operator readback for Reading.
+`ReadingOperationalTelemetryRepairReadback` is the package-owned seam that
+turns Reading pipeline contracts, storage projections, settlement facts, UI
+stage states, and repair posture into source-safe operator telemetry. Every
+phase, PTRR agent, PTRR step, `FailsafeGenerationSequence`,
+`ThricifiedGeneration`, `ToolExecution`, storage, ledger, wallet, delivery,
+UI, and repair event is proof-rooted and carries the execution-state metadata
+needed by the rich log component. Prompt visibility is template-id only;
+result visibility is parsed shape and proof roots only; raw interpolated
+prompts and raw provider responses stay absent. Repair hooks include rich-log
+inspection, source-safe metadata inspection, Need resynthesis, Depository
+search repair, BTC finality observation, projection repair, and pull-request
+delivery recovery.
+
+The operator readback persists under `reading/operational` as readback,
+operatorReadback, streamEvents, runbookHooks, telemetryRoot, repairRoot, and
+readbackRoot records. Gate 8 also extends the shared execution log to accept
+direct operational telemetry payloads so live Reading runs can display a
+compact row-by-row stream while preserving expandable source-safe metadata.
 
 ## Gate 9 implementation notes
 

BITCODE_SPEC_V39_NOTES.md

@@ -76,7 +76,7 @@ No `_legacy/` source is active source truth.
 | ReadFitsFinding runtime and replay | Gate 5 | `packages/pipelines/asset-pack/src/read-fits-finding-runtime.ts`, `depository-search.ts`, `.bitcode/v39-read-fits-finding-runtime.json`, package tests, protocol tests | implemented | Finding Fits searches the whole available Depository for many above-threshold candidates and persists source-safe replay, storage, telemetry, and repair receipts. |
 | AssetPack preview and quote boundary | Gate 6 | `packages/pipelines/asset-pack/src/asset-pack-preview-boundary.ts`, `.bitcode/v39-assetpack-preview-quote-boundary.json`, package tests, protocol tests | implemented | Preview exposes source-safe measurements, deterministic quote, settlement instructions, and withheld delivery posture without source-bearing AssetPack content. |
 | Settlement, BTD rights, and delivery | Gate 7 | `packages/pipelines/asset-pack/src/asset-pack-settlement-rights-delivery.ts`, `.bitcode/v39-settlement-rights-delivery.json`, BTD receipt/source-to-shares/reconciliation primitives, package tests, protocol tests | implemented | Payment unlocks rights and delivery only when finality, source-to-shares, BTD rights, reconciliation, and pull-request delivery agree; repair paths are auditable. |
-| Operational telemetry and repair | Gate 8 | stream events, operator readback, runbook hooks, proof roots, repair commands | pending | Reading is observable and repairable end to end. |
+| Operational telemetry and repair | Gate 8 | `ReadingOperationalTelemetryRepairReadback`, `.bitcode/v39-operational-telemetry-repair-readback.json`, package tests, UI tests, protocol tests | implemented | Reading is observable and repairable end to end through source-safe stream events, operator readback, runbook hooks, proof roots, and repair commands. |
 | Interface and Conversation parity | Gate 9 | Conversation, MCP/API, ChatGPT App, package contract tests | pending | Interfaces follow Terminal authority and cannot bypass gating, preview, settlement, rights, or delivery boundaries. |
 | Local and staging rehearsal | Gate 10 | local/staging lanes, real-inference gates, depository search, preview, settlement/delivery posture | pending | The full Reading flow rehearses in local and staging-testnet with production-mainnet value-bearing admission blocked. |
 | Promotion readiness | Gate 11 | V39 promotion report, generated proof appendix, promotion workflow, command dry-run | pending | V39 promotes only after all commercial Reading gates and source-safety evidence close. |
@@ -98,6 +98,7 @@ No `_legacy/` source is active source truth.
 | ReadFitsFinding runtime | `ReadFitsFindingSynthesis` searches many above-threshold deposits with replayable query, ranking, threshold, and selected-fit provenance receipts | implemented |
 | Preview and quote | AssetPack preview is source-safe and quote is deterministic before settlement | implemented |
 | Settlement and delivery | BTC settlement, BTD rights transfer, source-to-shares compensation, ledger/database/storage sync, and delivery are auditable | implemented |
+| Operational telemetry and repair | Reading stream events, operator readback, rich log rendering, proof roots, and repair hooks are source-safe and replayable | implemented |
 | Telemetry and repair | Reading emits source-safe rich stream events and operator readback with proof roots and repair posture | pending |
 | Interface parity | Conversation, MCP/API, ChatGPT App, and package consumers cannot bypass Terminal Reading authority | pending |
 | Local/staging rehearsal | Full Reading flow rehearses locally and in staging-testnet with mainnet value-bearing admission blocked | pending |
@@ -218,6 +219,23 @@ or unpaid AssetPack source. The generated proof artifact is
 `.bitcode/v39-settlement-rights-delivery.json` and the gate check is
 `pnpm run check:v39-gate7`.
 
+## Gate 8 Parity
+
+V39 Gate 8 is implemented when
+`ReadingOperationalTelemetryRepairReadback` wraps the Reading runtime,
+preview, settlement, storage, UI, and repair posture into source-safe
+operator readback. The parity source is
+`packages/pipelines/asset-pack/src/reading-operational-telemetry-repair-readback.ts`.
+It emits event kinds for phase, PTRR agent, PTRR step,
+`FailsafeGenerationSequence`, `ThricifiedGeneration`, `ToolExecution`,
+storage, ledger, wallet, delivery, UI, and repair. It persists
+`reading/operational` readback, stream events, runbook hooks, telemetry root,
+repair root, and readback root. The shared execution log accepts direct
+Reading telemetry payloads and renders event/proof/disclosure metadata without
+source leakage. The generated proof artifact is
+`.bitcode/v39-operational-telemetry-repair-readback.json` and the gate check
+is `pnpm run check:v39-gate8`.
+
 ## completion condition
 
-V39 Gate 7 is complete when `check:v39-gate7`, focused AssetPack settlement rights delivery package tests, V39 draft spec-family validation, V38/V39 canon-posture drift validation, promoted V38 spec-family validation, and Gate Quality pass on a `v39/gate-7-*` branch.
+V39 Gate 8 is complete when `check:v39-gate8`, focused AssetPack operational telemetry package tests, focused rich execution log UI tests, V39 draft spec-family validation, V38/V39 canon-posture drift validation, promoted V38 spec-family validation, and Gate Quality pass on a `v39/gate-8-*` branch.

BITCODE_SPEC_V39_PARITY_MATRIX.md

@@ -301,6 +301,15 @@ private settlement payloads, credentials, raw protected prompts, raw provider
 responses, or unpaid AssetPack source. Use
 `pnpm run generate:v39-settlement-rights-delivery` and
 `pnpm run check:v39-gate7` before closing the gate.
+V39 Gate 8 adds the package-owned
+`ReadingOperationalTelemetryRepairReadback` and source-safe generated artifact
+`.bitcode/v39-operational-telemetry-repair-readback.json`. Reading now
+projects phase, PTRR agent, PTRR step, Failsafe, ThricifiedGeneration,
+ToolExecution, storage, ledger, wallet, delivery, UI, and repair events into
+operator readback with proof roots, runbook hooks, prompt/result disclosure
+posture, and rich execution-log rendering. Use
+`pnpm run generate:v39-operational-telemetry-repair-readback` and
+`pnpm run check:v39-gate8` before closing the gate.
 V36 Gate 2 anchors market-wide activity through the package-owned
 `ExchangeActivityBook` and the source-safe generated artifact
 `.bitcode/v36-exchange-activity-book.json`, including listing, bid, ask,
@@ -468,7 +477,11 @@ through `pnpm run check:v39-gate4`; V39 Gate 5 is wired through
 `pnpm run check:v39-gate5` and its generated
 `.bitcode/v39-read-fits-finding-runtime.json` artifact. V39 Gate 6 is wired
 through `pnpm run check:v39-gate6` and its generated
-`.bitcode/v39-assetpack-preview-quote-boundary.json` artifact.
+`.bitcode/v39-assetpack-preview-quote-boundary.json` artifact. V39 Gate 7 is
+wired through `pnpm run check:v39-gate7` and its generated
+`.bitcode/v39-settlement-rights-delivery.json` artifact. V39 Gate 8 is wired
+through `pnpm run check:v39-gate8` and its generated
+`.bitcode/v39-operational-telemetry-repair-readback.json` artifact.
 V38 Gates 1 through 11 are wired through `pnpm run check:v38-gate1`,
 `pnpm run check:v38-gate2`, `pnpm run check:v38-gate3`,
 `pnpm run check:v38-gate4`, `pnpm run check:v38-gate5`,

README.md

@@ -5,8 +5,8 @@
 - Current active canonical pointer: `BITCODE_SPEC.txt` -> `V38`
 - Current active canon: `BITCODE_SPEC_V38.md`
 - Current draft target: `BITCODE_SPEC_V39.md`.
-- Current working gate: V39 Gate 7 Settlement, BTD Rights Transfer, And Delivery.
-- Next queued gate after V39 Gate 7 closure: V39 Gate 8 Operational Telemetry, Repair, And Operator Readback.
+- Current working gate: V39 Gate 8 Operational Telemetry, Repair, And Operator Readback.
+- Next queued gate after V39 Gate 8 closure: V39 Gate 9 Interface And Conversation Product Parity.
 - Latest closed version: V38 Inference Correctness, which promoted pipeline execution call-stack law, PTRR/Failsafe/Thricified parity, prompt and PromptPart benchmarking, source-safe inference telemetry, ReadNeedComprehensionSynthesis hardening, ReadFitsFindingSynthesis search and embeddings, AssetPack economic traceability, local/staging inference rehearsal, and V38 promotion readiness.
 - Recent V38 closure anchor: V38 Gate 11 Promotion Readiness generated V38 proof support, promoted `BITCODE_SPEC.txt` to `V38`, prepared V38 active / V39 draft runtime posture, and closed inference correctness canon.
 - V34 Gate 2 closure anchor: deployment-depth now owns package-backed `DeploymentHostCapabilityCatalog` and `EnvironmentLaneContract` source, deterministic `.bitcode/v34-deployment-host-capability-catalog.json` and `.bitcode/v34-environment-lane-contracts.json`, and visible `value-bearing-mainnet` blocking through `blocked_future_canon_required`.
@@ -67,6 +67,7 @@
 - V39 Gate 5 closure anchor: Commercial Reading readiness now owns package-backed `ReadFitsFindingRuntime` source, deterministic `.bitcode/v39-read-fits-finding-runtime.json`, accepted-Need-only Finding Fits admission, source-safe query plans, seven search channels, many-candidate ranking, selected-fit provenance roots, active OpenAI embedding policy, replay receipts, repair posture, package tests, protocol tests, workflow wiring, and `check:v39-gate5`.
 - V39 Gate 6 closure anchor: Reading readiness now owns package-backed `AssetPackPreviewBoundary` source, deterministic `.bitcode/v39-assetpack-preview-quote-boundary.json`, source-safe preview measurements, deterministic share-to-fee BTC quotes, disclosure leak scanning, selected-fit provenance, settlement instructions, withheld pull-request delivery posture, replay receipts, repair posture, package tests, protocol tests, workflow wiring, and `check:v39-gate6`.
 - V39 Gate 7 closure anchor: Reading readiness now owns package-backed `AssetPackSettlementRightsDeliveryBoundary` source, deterministic `.bitcode/v39-settlement-rights-delivery.json`, BTC payment observation/finality, source-to-shares contributor compensation, BTD rights transfer/read receipts, settlement unlock readback, ledger/database/object-storage reconciliation, post-settlement pull-request delivery unlock, repair posture, package tests, protocol tests, workflow wiring, and `check:v39-gate7`.
+- V39 Gate 8 closure anchor: Reading readiness now owns package-backed `ReadingOperationalTelemetryRepairReadback` source, deterministic `.bitcode/v39-operational-telemetry-repair-readback.json`, source-safe phase/PTRR agent/PTRR step/Failsafe/ThricifiedGeneration/tool/storage/ledger/wallet/delivery/UI/repair events, operator readback, proof roots, runbook hooks, rich execution log rendering, package tests, UI tests, protocol tests, workflow wiring, and `check:v39-gate8`.
 - Purpose: concise running index of Bitcode/ENGI specification history, current work, and planned work.
 
 This roadmap is not an active system specification.

SPECIFICATIONS_ROADMAP.md

@@ -233,6 +233,9 @@
     "generate:v39-settlement-rights-delivery": "node scripts/generate-v39-settlement-rights-delivery.mjs",
     "check:v39-settlement-rights-delivery": "node scripts/generate-v39-settlement-rights-delivery.mjs --check",
     "check:v39-gate7": "node scripts/check-v39-gate7-settlement-rights-delivery.mjs",
+    "generate:v39-operational-telemetry-repair-readback": "node scripts/generate-v39-operational-telemetry-repair-readback.mjs",
+    "check:v39-operational-telemetry-repair-readback": "node scripts/generate-v39-operational-telemetry-repair-readback.mjs --check",
+    "check:v39-gate8": "node scripts/check-v39-gate8-operational-telemetry-repair-readback.mjs",
     "generate:v38-inference-surface-inventory": "node scripts/generate-v38-inference-surface-inventory.mjs",
     "check:v38-inference-surface-inventory": "node scripts/generate-v38-inference-surface-inventory.mjs --check",
     "check:v38-gate2": "node scripts/check-v38-gate2-inference-surface-inventory.mjs",