Merge pull request #16 from engineeredsoftware/v28/gate-11-read-fits-finding-preview

Gate 11: close Finding Fits preview

Author: geraldarthurdavis

.github/workflows/bitcode-gate-quality.yml

@@ -47,6 +47,7 @@ jobs:
           node scripts/check-v28-metadevelopment-readiness.mjs --skip-branch-check
           node scripts/check-v28-gate9-depository-evidence.mjs
           node scripts/check-v28-gate10-read-need-comprehension.mjs
+          node scripts/check-v28-gate11-read-fits-finding-preview.mjs
           node scripts/promote-bitcode-canon.mjs --version V28 --commit HEAD --dry-run
 
       - name: Validate source casing and imports

BITCODE_SPEC_V28.md

@@ -850,6 +850,31 @@ Gate 11 owns `ReadFitsFindingSynthesis`: depository search, candidate fit
 ranking, fitting deposits as synthesis context, AssetPack measurement, source-
 safe preview, Share-to-Fee quote, and proof-rich Terminal stream visibility.
 
+Gate 11 acceptance requires:
+
+- discovery records all `fitDepositAssetIds` above the review and semantic
+  thresholds, preserves the ranked candidate trace, and fails closed with
+  `blocked_readiness` or `no_worthy_fit` instead of inventing source outside
+  the depository;
+- lexical and vector depository search tool telemetry use canonical
+  `ReadFitsFindingSynthesis.tool.*` ids and carry input read constraints,
+  selected fit deposits, query root, ranking root, and embedding policy posture;
+- implementation and postprocess carry the Finding Fits result into a
+  `bitcode.asset-pack.source-safe-preview` object with Need/Fit measurements,
+  fit score band, candidate ids, proof roots, source-manifest root, preview
+  root, access policy, settlement boundary, and unlock posture;
+- Share-to-Fee quote is deterministic:
+  `sum(measurement.weight * measurement.volume * admitted_fit_quality)` over
+  the accepted Need measurement vector and the admitted fit quality, with BTC
+  sats, quote root, payer, and network-fee posture visible before settlement;
+- protected source, full patches, source-bearing manifest entries, and licensed
+  read payloads stay withheld before settlement. Terminal may show only
+  measurements, non-source metadata, roots, candidate ids, proof posture,
+  ownership boundary, settlement boundary, and fee quote;
+- Terminal stream/history surfaces render the source-safe preview and fee quote
+  without requiring Network-panel JSON inspection, while preserving expandable
+  raw evidence for audit.
+
 ### Gate 12: Settlement, Rights, Delivery, And Reconciliation
 
 Gate 12 owns BTC settlement, depositor/reader ownership boundaries, read-license

BITCODE_SPEC_V28_PARITY_MATRIX.md

@@ -184,6 +184,18 @@ by a later V28 gate before `version/v28` can be promoted to `main`.
 | Full demonstration proof-suite alignment | Gate 13 | Refresh the older demonstration proof-member, theorem, public-projection, and V26-proven expectations so they match the active V27/V28 posture or are explicitly retired from V28 promotion proof. | `npm --prefix protocol-demonstration run test`, proof-member/theorem matrix counts, public projection spine assertions, and V26/V28 proven-generator assertions. | The complete demonstration suite is either green under the promoted V28 proof model or every intentionally retired legacy assertion is replaced by a narrower current-canon proof. |
 | Local and staging-testnet validation | Gate 13 | Local non-mocked OpenAI/Supabase/Vercel Sandbox validation plus staging-testnet readback evidence for complete Depositing -> Reading -> settlement -> PR delivery. | `check:v28-metadevelopment`, package tests, UAPI tests, build, demonstration tests, `qa:v28:pipeline-readback`, browser QA, SQL ledger/readback queries. | `version/v28` is promotion-ready only when local and staging-testnet evidence prove the full commercial product flow. |
 
+Gate 11 closure narrows the two Gate 11 carryforward rows to source-safe
+preview readiness: Finding Fits discovery must now produce canonical
+`ReadFitsFindingSynthesis.tool.lexical-depository-search` and
+`ReadFitsFindingSynthesis.tool.vector-depository-search` telemetry, all
+threshold-passing `fitDepositAssetIds`, query/ranking roots, and embedding
+policy posture. AssetPack postprocess must derive a source-safe preview and
+Share-to-Fee quote from the accepted Need and Finding Fits result, exposing only
+measurements, roots, score bands, candidate ids, proof posture, ownership
+boundary, settlement boundary, and BTC quote before settlement. Gate 12 remains
+responsible for paid unlock, rights transfer, ledger reconciliation, and PR
+delivery.
+
 ## V28 implementation matrix
 
 | Area | Current source evidence | Judgment | Gate owner |

BITCODE_V28_QA.md

@@ -2103,3 +2103,44 @@ Expected V28 behavior:
 - a later background revalidation may make the BTC/BTD widget show its own loading posture, but it must not regress to the disconnected CTA while a known wallet exists;
 - client telemetry must include `nav:chrome-identity`, `user-data:fetch-start`, `user-data:read` or `user-data:anonymous-read`, and any fetch failure;
 - server telemetry must include `auxillaries-data:read-start` and `auxillaries-data:read-finish` or `auxillaries-data:read-failed`.
+
+### Gate 11 Finding Fits And Source-Safe Preview QA
+
+Gate 11 local closure proves that `ReadFitsFindingSynthesis` can search the
+depository, preserve every qualifying fit deposit, derive a source-safe preview,
+and expose a deterministic Share-to-Fee quote before Gate 12 settlement unlock.
+
+Required local checks:
+
+```bash
+pnpm run check:v28-gate11
+pnpm --filter @bitcode/pipeline-asset-pack exec jest --config jest.config.cjs --runTestsByPath \
+  src/__tests__/depository-search.test.ts \
+  src/__tests__/read-need.test.ts \
+  src/__tests__/postprocess.test.ts \
+  src/__tests__/depository-search-tool.test.ts \
+  --runInBand --forceExit
+pnpm --filter @bitcode/pipeline-asset-pack typecheck
+```
+
+Expected evidence:
+
+- Finding Fits discovery stores `depository/search.result`,
+  `depository/search.toolTelemetry`, `fit.result`, `fit.selectionTrace`, all
+  threshold-passing `fitDepositAssetIds`, `queryRoot`, `rankingRoot`, and the
+  OpenAI `text-embedding-3-small` embedding policy posture.
+- Tool telemetry uses canonical
+  `ReadFitsFindingSynthesis.tool.lexical-depository-search` and
+  `ReadFitsFindingSynthesis.tool.vector-depository-search` ids. Vector
+  telemetry must at minimum expose the declared vector-store and embedding
+  policy posture until a later gate replaces local lexical fallback with live
+  vector RPC readback.
+- AssetPack postprocess stores `asset-pack/preview.sourceSafe`,
+  `asset-pack/preview.feeQuote`, and `asset-pack/preview.previewRoot` whenever
+  an accepted Need and Finding Fits result are present.
+- The source-safe preview exposes Need/Fit measurements, score band, fit
+  deposit ids, selected candidate ids, roots, proof posture, access policy,
+  ownership boundary, settlement boundary, unlock posture, and BTC quote.
+- The preview must not expose protected source content, full patches,
+  source-bearing manifest entries, licensed read payloads, or PR source changes
+  before settlement. Gate 12 owns paid unlock, rights transfer, and delivery.

package.json

@@ -41,6 +41,7 @@
     "check:v28-metadevelopment": "node scripts/check-v28-metadevelopment-readiness.mjs",
     "check:v28-gate9": "node scripts/check-v28-gate9-depository-evidence.mjs",
     "check:v28-gate10": "node scripts/check-v28-gate10-read-need-comprehension.mjs",
+    "check:v28-gate11": "node scripts/check-v28-gate11-read-fits-finding-preview.mjs",
     "check:spec-quality": "node scripts/run-bitcode-spec-quality.mjs --mode basic",
     "check:spec-quality:title": "node scripts/run-bitcode-spec-quality.mjs --mode strict-from-title",
     "check:spec-quality:v24": "node scripts/run-bitcode-spec-quality.mjs --mode strict-version --version V24",

packages/pipelines/asset-pack/src/__tests__/depository-search.test.ts

@@ -292,6 +292,36 @@ describe('AssetPack depository search', () => {
     expect(findStored(exec, 'fit', 'resultState')).toBe('worthy_fit');
     expect(findStored(exec, 'depository/search', 'result')?.resultState).toBe('worthy_fit');
     expect(findStored(exec, 'depository/search', 'embeddingPolicy')?.dimensions).toBe(1536);
+    expect(findStored(exec, 'depository/search', 'toolTelemetry')).toEqual([
+      expect.objectContaining({
+        tool: 'ReadFitsFindingSynthesis.tool.lexical-depository-search',
+        phase: 'ReadFitsFindingSynthesis.discovery',
+        agent: 'ReadFitsFindingSynthesis.discovery.finding-fits',
+        step: 'ReadFitsFindingSynthesis.discovery.finding-fits.try',
+        output: expect.objectContaining({
+          resultState: 'worthy_fit',
+          fitDepositAssetIds: ['asset_repository-revision-deposit-engineeredsoftware-engi'],
+          queryRoot: expect.stringMatching(/^sha256:/),
+          rankingRoot: expect.stringMatching(/^sha256:/),
+        }),
+      }),
+      expect.objectContaining({
+        tool: 'ReadFitsFindingSynthesis.tool.vector-depository-search',
+        output: expect.objectContaining({
+          resultState: 'embedding_policy_declared',
+          vectorStore: expect.objectContaining({
+            table: 'deliverable_vectors',
+            rpc: 'match_deliverable_vectors',
+          }),
+        }),
+      }),
+    ]);
+    expect(findStored(exec, 'tools', 'lexical-depository-search')?.tool).toBe(
+      'ReadFitsFindingSynthesis.tool.lexical-depository-search'
+    );
+    expect(findStored(exec, 'tools', 'vector-depository-search')?.tool).toBe(
+      'ReadFitsFindingSynthesis.tool.vector-depository-search'
+    );
     expect(output.fitResult.resultState).toBe('worthy_fit');
     expect(output.fitResult.fitDepositAssetIds).toEqual([
       'asset_repository-revision-deposit-engineeredsoftware-engi',

packages/pipelines/asset-pack/src/__tests__/postprocess.test.ts

@@ -1,6 +1,7 @@
 // @ts-nocheck
 import { buildAssetPackPostprocessedResult, normalizeAssetPackOutput } from '../postprocess';
 import { Execution } from '@bitcode/execution-generics';
+import { acceptReadNeed, synthesizeReadNeedForPipelineInput } from '../read-need';
 
 describe('normalizeAssetPackOutput', () => {
   it('backfills prUrl, filesModified, and summary from execution', () => {
@@ -138,4 +139,94 @@ describe('normalizeAssetPackOutput', () => {
       prUrl: 'https://github.com/acme/repo/pull/26',
     });
   });
+
+  it('derives and stores source-safe preview evidence from an accepted Need and Finding Fits result', () => {
+    const exec = new Execution('pipeline:asset-pack');
+    const acceptedNeed = acceptReadNeed(synthesizeReadNeedForPipelineInput({
+      read: {
+        id: 'read-1',
+        prompt: 'Find deposited source evidence for a source-safe Terminal AssetPack preview.',
+      },
+      sourceRevision: {
+        repositoryFullName: 'engineeredsoftware/ENGI',
+        branch: 'main',
+        commit: '31bbc0c5227b6b3aed5d107fd8507d35ec22970a',
+      },
+    }));
+    const fitResult = {
+      schema: 'bitcode.asset-pack.fit-result',
+      resultState: 'worthy_fit',
+      resultReasons: ['Selected 1 proof-bearing fit deposit for this Read.'],
+      fitDepositAssetIds: ['fit-deposit-1'],
+      selectedCandidateAssetIds: ['fit-deposit-1'],
+      queryRoot: 'sha256:query',
+      rankingRoot: 'sha256:ranking',
+      searchedAssetCount: 1,
+      embeddingPolicy: {
+        provider: 'openai',
+        model: 'text-embedding-3-small',
+        dimensions: 1536,
+      },
+      selectionTrace: {
+        selectedCandidates: [
+          {
+            assetId: 'fit-deposit-1',
+            scores: { finalScore: 0.84 },
+            proofEvidence: { proofRoot: 'sha256:proof' },
+          },
+        ],
+        fitDeposits: [],
+        blockedCandidates: [],
+        candidateRanking: [],
+        rejectedCandidateCount: 0,
+      },
+    };
+
+    exec.store('read/need', 'accepted', acceptedNeed);
+    exec.store('fit', 'result', fitResult);
+
+    const normalized = normalizeAssetPackOutput({
+      success: true,
+      summary: 'Measured AssetPack preview ready.',
+      deliveryMechanism: {
+        prUrl: 'https://github.com/engineeredsoftware/ENGI/pull/28',
+      },
+    } as any, exec);
+    const result = buildAssetPackPostprocessedResult(exec, normalized);
+
+    expect(normalized.sourceSafePreview).toMatchObject({
+      schema: 'bitcode.asset-pack.source-safe-preview',
+      need: {
+        needId: acceptedNeed.needId,
+        reviewState: 'accepted',
+      },
+      fit: {
+        resultState: 'worthy_fit',
+        fitDepositAssetIds: ['fit-deposit-1'],
+        scoreBand: 'high',
+      },
+      disclosurePolicy: {
+        protectedSourceDisclosure: 'forbidden_before_settlement',
+      },
+      settlementBoundary: {
+        payer: 'reader',
+        payee: 'depositor',
+        serverCustody: false,
+      },
+      unlock: {
+        state: 'pending_settlement',
+        sourceAvailable: false,
+      },
+    });
+    expect(normalized.sourceSafePreview.delivery.pullRequestTarget).toBe(
+      'https://github.com/engineeredsoftware/ENGI/pull/28'
+    );
+    expect(normalized.feeQuote.quoteRoot).toMatch(/^sha256:/);
+    expect(result.sourceSafePreview?.roots.previewRoot).toMatch(/^sha256:/);
+    expect(result.feeQuote?.finalityState).toBe('preview_not_paid');
+    expect(exec.get('asset-pack/preview', 'sourceSafe')?.previewId).toBe(
+      normalized.sourceSafePreview.previewId
+    );
+    expect(JSON.stringify(normalized.sourceSafePreview)).not.toContain('diff --git');
+  });
 });

packages/pipelines/asset-pack/src/depository-search.ts

@@ -269,6 +269,11 @@ const DEFAULT_THRESHOLDS: DepositorySearchThresholds = {
   maxSelectedCandidates: 3,
 };
 
+export const READ_FITS_FINDING_SYNTHESIS_TOOL_IDS = {
+  lexicalDepositorySearch: 'ReadFitsFindingSynthesis.tool.lexical-depository-search',
+  vectorDepositorySearch: 'ReadFitsFindingSynthesis.tool.vector-depository-search',
+} as const;
+
 const STOP_WORDS = new Set([
   'the',
   'and',
@@ -1280,42 +1285,67 @@ function storeDepositorySearchToolResult(
 ): void {
   if (!execution?.store) return;
   const { read, assets, result, providerIds } = input;
-  execution.store('tools', 'result', {
-    tool: 'bitcode.depository.search',
-    ok: true,
-    input: {
-      read: {
-        id: read.id || null,
-        repositoryFullName: read.repositoryFullName || null,
-        sourceBranch: read.sourceBranch || null,
-        sourceCommit: read.sourceCommit || null,
-        targetArtifactKinds: read.targetArtifactKinds,
-        closureCriteriaCount: read.closureCriteria.length,
-        failureModeCount: read.failureModes.length,
-      },
-      assetCount: assets.length,
-      providerIds,
+  const toolInput = {
+    read: {
+      id: read.id || null,
+      repositoryFullName: read.repositoryFullName || null,
+      sourceBranch: read.sourceBranch || null,
+      sourceCommit: read.sourceCommit || null,
+      targetArtifactKinds: read.targetArtifactKinds,
+      closureCriteriaCount: read.closureCriteria.length,
+      failureModeCount: read.failureModes.length,
     },
+    assetCount: assets.length,
+    providerIds,
+  };
+  const toolOutput = {
+    schema: result.schema,
+    resultState: result.resultState,
+    resultReasons: result.resultReasons,
+    searchedAssetCount: result.searchedAssetCount,
+    fitDepositAssetIds: result.fitDepositAssetIds,
+    fitDepositCount: result.fitDeposits.length,
+    selectedCandidateAssetIds: result.selectedCandidateAssetIds,
+    selectedCandidateCount: result.selectedCandidates.length,
+    blockedCandidateCount: result.blockedCandidates.length,
+    rejectedCandidateCount: result.rejectedCandidates.length,
+    queryRoot: result.queryRoot,
+    rankingRoot: result.rankingRoot,
+    embeddingPolicy: result.embeddingPolicy,
+  };
+  const lexicalTelemetry = {
+    tool: READ_FITS_FINDING_SYNTHESIS_TOOL_IDS.lexicalDepositorySearch,
+    ok: true,
+    input: toolInput,
+    output: toolOutput,
+    phase: 'ReadFitsFindingSynthesis.discovery',
+    agent: 'ReadFitsFindingSynthesis.discovery.finding-fits',
+    step: 'ReadFitsFindingSynthesis.discovery.finding-fits.try',
+    generation: 'tools_execution',
+  };
+  const vectorTelemetry = {
+    tool: READ_FITS_FINDING_SYNTHESIS_TOOL_IDS.vectorDepositorySearch,
+    ok: true,
+    input: toolInput,
     output: {
-      schema: result.schema,
-      resultState: result.resultState,
-      resultReasons: result.resultReasons,
-      searchedAssetCount: result.searchedAssetCount,
-      fitDepositAssetIds: result.fitDepositAssetIds,
-      fitDepositCount: result.fitDeposits.length,
+      resultState: 'embedding_policy_declared',
       selectedCandidateAssetIds: result.selectedCandidateAssetIds,
-      selectedCandidateCount: result.selectedCandidates.length,
-      blockedCandidateCount: result.blockedCandidates.length,
-      rejectedCandidateCount: result.rejectedCandidates.length,
+      fitDepositAssetIds: result.fitDepositAssetIds,
       queryRoot: result.queryRoot,
       rankingRoot: result.rankingRoot,
       embeddingPolicy: result.embeddingPolicy,
+      vectorStore: result.embeddingPolicy.vectorStore,
     },
-    phase: 'setup',
-    agent: 'setup:depository-search',
-    step: 'try',
+    phase: 'ReadFitsFindingSynthesis.discovery',
+    agent: 'ReadFitsFindingSynthesis.discovery.finding-fits',
+    step: 'ReadFitsFindingSynthesis.discovery.finding-fits.try',
     generation: 'tools_execution',
-  });
+  };
+
+  execution.store('tools', 'result', lexicalTelemetry);
+  execution.store('tools', 'lexical-depository-search', lexicalTelemetry);
+  execution.store('tools', 'vector-depository-search', vectorTelemetry);
+  execution.store('depository/search', 'toolTelemetry', [lexicalTelemetry, vectorTelemetry]);
 }
 
 function buildBlockedReadFitsFindingResult(input: {
@@ -1406,12 +1436,17 @@ export async function runDepositorySearchForPipelineInput(
   if (execution?.store) {
     storeEvidence(execution);
     storeEvidence(execution.parent as { store?: (namespace: string, key: string, value: unknown) => void });
-    storeDepositorySearchToolResult(execution, {
+    const toolEvidence = {
       read,
       assets,
       result,
       providerIds: providers.map((provider) => provider.id),
-    });
+    };
+    storeDepositorySearchToolResult(execution, toolEvidence);
+    storeDepositorySearchToolResult(
+      execution.parent as { store?: (namespace: string, key: string, value: unknown) => void },
+      toolEvidence
+    );
   }
 
   return result;