Gate 4: Reading pipeline observability (#23)

* Gate 4: project Reading pipeline observability

Add contract-aware observability inventory and projection for ReadNeedComprehensionSynthesis and ReadFitsFindingSynthesis, stamp sandbox harness stream events with PTRR and ThricifiedGeneration metadata, and surface those fields through the Terminal execution stream UI.

Update V29 spec, parity, docs, Gate 4 checker, CI wiring, and focused package/UAPI tests for the Reading telemetry closure criteria.

* Gate 4: satisfy UAPI observability typecheck

Normalize PTRR step-id parsing so the stricter UAPI TypeScript build receives a concrete string before checking Reading pipeline telemetry step names.

Validation: pnpm -C uapi exec tsc --noEmit; pnpm run check:v29-gate4; pnpm --dir uapi exec jest --runTestsByPath tests/terminalPipelineHarnessClient.test.ts tests/pipelineExecutionLogHeader.test.tsx --runInBand; pnpm exec eslint packages/pipelines/asset-pack/src/reading-pipeline-observability.ts uapi/app/terminal/terminal-pipeline-harness-client.ts uapi/components/base/bitcode/execution/pipeline-execution-log-header.tsx uapi/components/base/bitcode/execution/pipeline-execution-log.tsx --ext .ts,.tsx; git diff --check

Author: geraldarthurdavis

.github/workflows/bitcode-gate-quality.yml

@@ -48,6 +48,7 @@ jobs:
           node scripts/check-v29-gate1-objectives-and-gating.mjs --skip-branch-check
           node scripts/check-v29-gate2-terminal-transaction-read-models.mjs --skip-branch-check
           node scripts/check-v29-gate3-wallet-signer-btc-operations.mjs --skip-branch-check
+          node scripts/check-v29-gate4-reading-pipeline-observability.mjs --skip-branch-check
 
       - name: Validate source casing and imports
         run: |
@@ -65,6 +66,7 @@ jobs:
           pnpm --filter @bitcode/btd test
           pnpm --filter @bitcode/btd test -- --runTestsByPath __tests__/btc-fee-operation.test.ts
           pnpm --filter @bitcode/pipeline-asset-pack exec jest --config jest.config.cjs --passWithNoTests --forceExit
+          pnpm --filter @bitcode/pipeline-asset-pack exec jest --config jest.config.cjs --runTestsByPath src/__tests__/reading-pipeline-observability.test.ts src/__tests__/reading-pipeline-contract.test.ts --runInBand
           pnpm --filter @bitcode/pipeline-hosts exec jest --config jest.config.cjs --passWithNoTests --forceExit
 
       - name: Test staged Reading route and Terminal harness contracts

BITCODE_SPEC_V29.md

@@ -247,6 +247,40 @@ It may not expose protected AssetPack source, wallet secrets, provider tokens, o
 - Current validating commands and parity basis: pipeline contract tests, route tests, prompt rendering/audit checks, local live OpenAI validation, and V29 gate checks.
 - Current accepted boundaries: full-profile async push completion can deepen in V29 gates but cannot bypass source-safe preview or settlement boundaries.
 
+#### V29 Reading pipeline observability canon
+
+Reading pipeline observability is contract-projected, not ad hoc.
+The asset-pack package owns the inventory for `ReadNeedComprehensionSynthesis` and `ReadFitsFindingSynthesis`, and live harness events must project back to that inventory whenever they are emitted.
+
+The observable contract levels are:
+
+- execution;
+- phase;
+- PTRR agent;
+- PTRR step;
+- ThricifiedGeneration;
+- prompt;
+- tool;
+- raw output;
+- parsed output.
+
+For every live Reading stream event that can be associated with a contract, Terminal-visible telemetry carries:
+
+- pipeline name;
+- phase id;
+- PTRR agent id;
+- PTRR step id and step name;
+- ThricifiedGeneration id and failsafe;
+- prompt template id and generation prompt ids;
+- tool id, input type, and output type when the event is tool-backed;
+- declared return type and output schema;
+- prompt template/interpolated prompt presence;
+- reasoning, judgment, raw model response, and parsed typed output presence.
+
+The Vercel Sandbox harness must export both the observability inventory and an observability coverage summary in its evidence artifact.
+Coverage readback is not a settlement proof by itself; it is the operational proof that Terminal can debug a live Reading run without relying on browser network logs.
+Failure to observe prompt, raw-output, parsed-output, or tool telemetry in a run that reaches those stages is a blocked-readiness signal for Gate 4+ QA.
+
 ### Fit, recall, ranking, and verification
 
 - Current canonical objects and emitted artifacts: FindingFits query root, lexical search result, vector search result, fit deposit list, candidate score, ranking root, blocker list, and verification decision.

BITCODE_SPEC_V29_DELTA.md

@@ -113,6 +113,15 @@ Closure acceptance:
 
 Gate 4 owns deep Terminal visibility over `ReadNeedComprehensionSynthesis` and `ReadFitsFindingSynthesis`, including execution, phase, PTRR agent, PTRR step, ThricifiedGeneration, prompt, tool, output, and typed parse telemetry.
 
+Closure acceptance:
+
+- `@bitcode/pipeline-asset-pack` exposes a Reading pipeline observability inventory and `ReadingPipelineTelemetryProjection` over the existing Reading pipeline contracts.
+- The inventory accounts for both Reading pipelines, 11 phases, 12 PTRR agents, 48 PTRR steps, 144 ThricifiedGenerations, all registered prompt templates, all ThricifiedGeneration prompt ids, and the four Finding Fits tools.
+- The Vercel Sandbox harness annotates stream events with pipeline name, phase id, PTRR agent id, PTRR step id, ThricifiedGeneration id, prompt template id, generation prompt ids, tool id, output schema, return type, and prompt/raw/parsed/tool evidence posture.
+- Harness evidence includes Reading observability inventory and coverage readback so local/staging failures show exactly which telemetry levels were not observed.
+- Terminal converts harness SSE events into the shared execution log shape with contract-aware execution state, summary rows, and expandable metadata for prompt template, interpolated prompt, raw response, parsed typed output, tools, and output schema.
+- Gate-quality CI runs the Gate 4 checker plus focused asset-pack, pipeline-host, Terminal harness, and execution-log header tests.
+
 ### Gate 5: AssetPack Disclosure Rights And Preview Depth
 
 Gate 5 owns AssetPack range detail, owner-read/licensed-read/denied-state flows, source-safe preview, disclosure policy review, paid unlock state, and protected-source leakage tests.

BITCODE_SPEC_V29_NOTES.md

@@ -60,6 +60,7 @@ For a simplified reading of V29:
 
 4. **Gate 4: Reading Transaction Recovery And Pipeline Observability**
    - Make `ReadNeedComprehensionSynthesis` and `ReadFitsFindingSynthesis` execution telemetry fully readable in Terminal at execution, phase, PTRR agent, PTRR step, ThricifiedGeneration, prompt, tool, raw output, and parsed output levels.
+   - Acceptance detail: the asset-pack package owns the observability inventory/projection; the sandbox harness stamps live stream events with contract identifiers and coverage; Terminal renders those identifiers in the shared execution stream header, row summary, and expandable metadata.
 
 5. **Gate 5: AssetPack Disclosure Rights And Preview Depth**
    - Deepen source-safe preview, AssetPack range detail, owner-read/licensed-read/denied state, disclosure policy review, paid unlock, and source-leakage tests.
@@ -120,6 +121,19 @@ Accepted Gate 3 posture:
 - Terminal has a Wallet/BTC detail section so fee readiness is visible before raw ledger payload inspection.
 - Gate 3 does not implement settlement reconciliation repair; Gate 6 owns drift repair and ledger/database reconciliation action depth.
 
+## Gate 4 working notes
+
+Gate 4 makes Reading pipeline telemetry readable without changing the underlying execution primitive names.
+Execution remains the base primitive, Reading pipelines compose phases, each phase owns PTRR agents, each PTRR agent owns plan/try/refine/retry steps, and each step carries ThricifiedGeneration prompt/output evidence.
+
+Accepted Gate 4 posture:
+
+- `reading-pipeline-contract.ts` remains the source of pipeline, phase, PTRR agent, PTRR step, ThricifiedGeneration, prompt, tool, and return-type contracts.
+- `reading-pipeline-observability.ts` projects live stream events onto those contracts and reports observability coverage.
+- The sandbox harness records contract-aware telemetry events and stores coverage in exported evidence.
+- Terminal's live harness stream shows pipeline, phase id, PTRR agent, PTRR step, ThricifiedGeneration, prompt, tool, output schema, raw output posture, and parsed typed output posture through the shared execution log components.
+- Gate 4 does not implement disclosure-depth unlock, settlement repair, organization authority, or full browser proof; later gates own those slices.
+
 ## Later-version boundaries
 
 - V30 remains reserved for Protocol/BTD hardening that is not necessary to close Terminal transaction depth.

BITCODE_SPEC_V29_PARITY_MATRIX.md

@@ -51,7 +51,7 @@ No `_legacy/` source is active source truth.
 | Workflow retargeting | Gate 1 | `.github/workflows/bitcode-gate-quality.yml`, `.github/workflows/bitcode-canon-quality.yml` | drafted | CI checks V28 active / V29 draft posture instead of stale V27/V28 posture. |
 | Terminal transaction read models | Gate 2 | `uapi/app/terminal/terminal-transaction-read-model.ts`, `uapi/app/terminal/terminal-transaction-query.ts`, `TerminalTransactionWorkspace.tsx`, `TerminalTransactionDetailSurface.tsx`, UAPI tests, Gate 2 checker | drafted | Terminal transaction state is URL-addressable, recoverable, typed, low-detail by default, and expandable without raw JSON as the ordinary operator contract. |
 | Wallet signer/BTC operations | Gate 3 | `packages/btd/src/btc-fee-operation.ts`, BTC fee route, Terminal Wallet/BTC detail section, BTD and UAPI tests, Gate 3 checker | drafted | Signer session, PSBT, broadcast/finality/reorg/replacement/failure states are ordinary Terminal states. |
-| Reading pipeline observability | Gate 4 | `packages/pipelines/asset-pack`, `packages/pipeline-hosts`, Terminal stream components | pending | Pipeline/phase/PTRR/ThricifiedGeneration/tool/prompt telemetry is complete and readable. |
+| Reading pipeline observability | Gate 4 | `packages/pipelines/asset-pack/src/reading-pipeline-observability.ts`, `packages/pipeline-hosts/src/asset-pack-harness.ts`, Terminal stream components, Gate 4 checker | drafted | Pipeline/phase/PTRR/ThricifiedGeneration/tool/prompt/raw-output/parsed-output telemetry is contract-projected, complete, and readable. |
 | AssetPack disclosure rights | Gate 5 | BTD access primitives, AssetPack preview UI, disclosure policy tests | pending | Source-safe preview and paid unlock are proven without source leakage. |
 | Settlement reconciliation repair | Gate 6 | BTD journal/reconciliation, Supabase readback, Terminal repair UI | pending | Ledger, database, and metaphysical state drift is visible and repairable. |
 | Organization permission authority | Gate 7 | access policy, org holdings, MCP/ChatGPT gates, Terminal permission UI | pending | Registry-derived roles, holdings, and read-license authority govern actions. |
@@ -69,6 +69,7 @@ No `_legacy/` source is active source truth.
 | Gate 1 script | `pnpm run check:v29-gate1` fails closed on stale posture or missing gates | drafted |
 | Gate 2 read model | `pnpm run check:v29-gate2` proves typed route-owned Terminal transaction reading | drafted |
 | Gate 3 wallet/BTC operation | `pnpm run check:v29-gate3` proves quote lifecycle, signer recovery, blocked readiness, API posture, and Terminal Wallet/BTC detail | drafted |
+| Gate 4 Reading observability | `pnpm run check:v29-gate4` proves contract-aware Reading stream telemetry and Terminal rendering | drafted |
 | Product implementation gates | Gates 3-9 close remaining Terminal transaction depth with tests and docs | pending |
 | Promotion gate | Gate 10 closes generated proof and promotion automation | pending |
 
@@ -139,3 +140,26 @@ Gate 2 is complete when Terminal selected transaction state rewrites recoverable
 ## Gate 3 completion condition
 
 Gate 3 is complete when package primitives model BTC quotes, signer recovery, operation posture, and blocked readiness; API responses serialize operation posture; Terminal has a Wallet/BTC route-owned detail section; focused package and UAPI tests pass; `check:v29-gate3` passes; CI invokes the checker and tests; docs name the implemented source surfaces; and the gate branch is committed and pushed for review into `version/v29`.
+
+## Gate 4 Parity
+
+| Requirement | Source evidence | Current V29 judgment |
+| --- | --- | --- |
+| Reading pipeline contracts remain the source of truth | `reading-pipeline-contract.ts`, contract tests | drafted |
+| Observability inventory covers both Reading pipelines and all contract levels | `reading-pipeline-observability.ts`, observability tests | drafted |
+| Live sandbox harness stamps stream events with contract ids and output posture | `packages/pipeline-hosts/src/asset-pack-harness.ts`, harness tests | drafted |
+| Harness evidence exports inventory and coverage readback | `readingPipelineObservabilityInventory`, `readingPipelineObservabilityCoverage`, harness tests | drafted |
+| Terminal summaries and execution state expose pipeline, phase, PTRR, ThricifiedGeneration, prompt, tool, and schema fields | `terminal-pipeline-harness-client.ts`, Terminal harness tests | drafted |
+| Shared execution log header renders contract-aware identifiers without raw JSON first | `pipeline-execution-log-header.tsx`, header tests | drafted |
+| Gate 4 checker is wired to package scripts and CI | `scripts/check-v29-gate4-reading-pipeline-observability.mjs`, `package.json`, gate workflow | drafted |
+
+## Gate 4 accepted boundaries
+
+- Gate 4 does not add a new execution primitive; it projects existing execution, pipeline, PTRR, and ThricifiedGeneration primitives.
+- Gate 4 does not expose protected AssetPack source before settlement.
+- Gate 4 does not close disclosure-depth unlock, settlement repair, organization authority, or full browser proof.
+- Gate 4 does not add versioned API routes or source identifiers.
+
+## Gate 4 completion condition
+
+Gate 4 is complete when Reading pipeline observability is package-owned, stream events are contract-projected in the sandbox harness, Terminal renders the projection through shared execution components, focused package and UAPI tests pass, `check:v29-gate4` passes, CI invokes the checker and tests, docs name the implemented source surfaces, and the gate branch is committed and pushed for review into `version/v29`.

package.json

@@ -47,6 +47,7 @@
     "check:v29-gate1": "node scripts/check-v29-gate1-objectives-and-gating.mjs",
     "check:v29-gate2": "node scripts/check-v29-gate2-terminal-transaction-read-models.mjs",
     "check:v29-gate3": "node scripts/check-v29-gate3-wallet-signer-btc-operations.mjs",
+    "check:v29-gate4": "node scripts/check-v29-gate4-reading-pipeline-observability.mjs",
     "check:spec-quality": "node scripts/run-bitcode-spec-quality.mjs --mode basic",
     "check:spec-quality:title": "node scripts/run-bitcode-spec-quality.mjs --mode strict-from-title",
     "check:spec-quality:v24": "node scripts/run-bitcode-spec-quality.mjs --mode strict-version --version V24",

packages/pipeline-hosts/src/__tests__/asset-pack-harness.test.ts

@@ -187,6 +187,14 @@ describe('asset-pack sandbox harness plan', () => {
     expect(source).toContain('rawModelResponsePresent');
     expect(source).toContain('parsedTypedOutputPresent');
     expect(source).toContain('inferenceAudit');
+    expect(source).toContain('buildReadingPipelineObservabilityInventory');
+    expect(source).toContain('resolveReadingPipelineTelemetryProjection');
+    expect(source).toContain('summarizeReadingPipelineObservabilityCoverage');
+    expect(source).toContain('readingPipelineTelemetry');
+    expect(source).toContain('ptrrStepId');
+    expect(source).toContain('thricifiedGenerationId');
+    expect(source).toContain('outputSchema');
+    expect(source).toContain('readingPipelineObservabilityCoverage');
     expect(source).toContain('Pipeline produced ');
     expect(source).toContain('sourceSafePreview,');
     expect(source).toContain('protectedSourceUnlock');

packages/pipeline-hosts/src/asset-pack-harness.ts

@@ -392,6 +392,9 @@ let checkpointTimer = null;
 let heartbeatTimer = null;
 let checkpointInFlight = Promise.resolve();
 let lastCheckpointAt = 0;
+let readingPipelineObservabilityInventory = null;
+let resolveReadingPipelineTelemetryProjectionFn = null;
+let summarizeReadingPipelineObservabilityCoverageFn = null;
 
 function normalizeResultState(candidate) {
   return ['worthy_fit', 'no_worthy_fit', 'blocked_readiness'].includes(candidate)
@@ -532,6 +535,15 @@ function summarizeStreamEvent(event) {
         parsedTypedOutput: summarizeInspectableValue(data.parsedTypedOutput ?? data.parsed ?? null),
       }
     : null;
+  const readingPipelineTelemetry = resolveReadingPipelineTelemetryProjectionFn
+    ? resolveReadingPipelineTelemetryProjectionFn({
+        ...event,
+        data: {
+          ...(data || {}),
+          telemetryEvent: event,
+        },
+      })
+    : null;
   return {
     type: 'pipeline-stream-event',
     stage: stageForStreamEvent(event),
@@ -558,6 +570,19 @@ function summarizeStreamEvent(event) {
     parsedTypedOutputPresent: Boolean(data?.parsedTypedOutput || data?.parsed),
     inferenceAudit: llmAudit,
     inspectable: summarizeLlmInspectable(event, event?.data ?? null),
+    readingPipelineTelemetry,
+    pipelineName: readingPipelineTelemetry?.pipelineName || null,
+    phaseId: readingPipelineTelemetry?.phaseId || null,
+    agentId: readingPipelineTelemetry?.agentId || null,
+    ptrrStepId: readingPipelineTelemetry?.ptrrStepId || null,
+    ptrrStepName: readingPipelineTelemetry?.ptrrStepName || null,
+    thricifiedGenerationId: readingPipelineTelemetry?.thricifiedGenerationId || null,
+    thricifiedFailsafe: readingPipelineTelemetry?.thricifiedFailsafe || null,
+    promptTemplateId: readingPipelineTelemetry?.promptTemplateId || null,
+    generationPromptIds: readingPipelineTelemetry?.generationPromptIds || null,
+    toolId: readingPipelineTelemetry?.toolId || null,
+    outputSchema: readingPipelineTelemetry?.outputSchema || null,
+    returnType: readingPipelineTelemetry?.returnType || null,
   };
 }
 
@@ -672,6 +697,10 @@ function checkpointEvidence(reason) {
     output,
     error,
     execution: execution ? summarizeExecution(execution) : null,
+    readingPipelineObservabilityInventory,
+    readingPipelineObservabilityCoverage: summarizeReadingPipelineObservabilityCoverageFn
+      ? summarizeReadingPipelineObservabilityCoverageFn(events)
+      : null,
     events,
     startedAt,
     checkpointAt: new Date().toISOString(),
@@ -1523,14 +1552,26 @@ try {
   manifestRoot = createHash('sha256').update(JSON.stringify(manifest)).digest('hex');
   userId = process.env.BITCODE_PIPELINE_USER_ID || manifest.deposit?.userId || DEFAULT_USER_ID;
   const [
-    { assetPackPipeline, acceptReadNeed, buildAssetPackSourceSafePreview, isAcceptedReadNeed, synthesizeReadNeedForPipelineInput },
+    {
+      assetPackPipeline,
+      acceptReadNeed,
+      buildAssetPackSourceSafePreview,
+      buildReadingPipelineObservabilityInventory,
+      isAcceptedReadNeed,
+      resolveReadingPipelineTelemetryProjection,
+      summarizeReadingPipelineObservabilityCoverage,
+      synthesizeReadNeedForPipelineInput,
+    },
     { enablePipelineStreaming, factoryPipelineExecution },
     { applyAssetPackSettlementUnlockToPreview, buildAssetPackSettlementUnlock },
   ] = await Promise.all([
     import('../../packages/pipelines/asset-pack/src/index'),
     import('../../packages/pipelines-generics/src/index'),
     import('../../packages/btd/src/settlement'),
   ]);
+  readingPipelineObservabilityInventory = buildReadingPipelineObservabilityInventory();
+  resolveReadingPipelineTelemetryProjectionFn = resolveReadingPipelineTelemetryProjection;
+  summarizeReadingPipelineObservabilityCoverageFn = summarizeReadingPipelineObservabilityCoverage;
   execution = factoryPipelineExecution('asset_pack', undefined, {
     pipelineName: 'asset_pack',
     family: 'asset_pack',
@@ -1709,6 +1750,9 @@ try {
     settlementAdmissible: ledgerSettlement.settlementAdmissible,
     assetPackId: ledgerSettlement.assetPackId || null,
   });
+  const readingPipelineObservabilityCoverage = summarizeReadingPipelineObservabilityCoverageFn
+    ? summarizeReadingPipelineObservabilityCoverageFn(events)
+    : null;
 
   const evidence = {
     schema: 'bitcode.pipeline-harness.evidence',
@@ -1730,6 +1774,8 @@ try {
     shippables: output?.shippables || null,
     ledgerSettlement: output.ledgerSettlement,
     execution: summarizeExecution(execution),
+    readingPipelineObservabilityInventory,
+    readingPipelineObservabilityCoverage,
     events,
     startedAt,
     completedAt: new Date().toISOString(),
@@ -1774,6 +1820,10 @@ try {
     output,
     error,
     execution: execution ? summarizeExecution(execution) : null,
+    readingPipelineObservabilityInventory,
+    readingPipelineObservabilityCoverage: summarizeReadingPipelineObservabilityCoverageFn
+      ? summarizeReadingPipelineObservabilityCoverageFn(events)
+      : null,
     events,
     startedAt,
     completedAt: new Date().toISOString(),

packages/pipelines/asset-pack/package.json

@@ -8,6 +8,7 @@
     ".": "./src/index.ts",
     "./read-need": "./src/read-need.ts",
     "./reading-pipeline-contract": "./src/reading-pipeline-contract.ts",
+    "./reading-pipeline-observability": "./src/reading-pipeline-observability.ts",
     "./depository-search": "./src/depository-search.ts"
   },
   "scripts": {