You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
V40 Gate 10: Prompt Benchmark Smoke And V41 Readiness
Adds source-safe PromptPart and composed Prompt benchmark smoke automation, generated protocol artifact, workflow wiring, and Gate 10 checks. Updates the V40 spec family, roadmap, and package documentation to hand focused prompt-program evolution to V41.
- Source parity state: V40 opens testing-system specification over active V39; package, API, pipeline, conversation, Terminal, Auxillaries, Exchange, interface, ledger/database/storage, and demonstration tests remain to be hardened gate by gate
14
14
- Notes companion: `BITCODE_SPEC_V40_NOTES.md`
15
15
- Delta companion: `BITCODE_SPEC_V40_DELTA.md`
@@ -160,6 +160,14 @@ The operator command `rehearse:v40-local-staging` builds local or staging-testne
160
160
Staging-testnet automation binds the Supabase project `tkpyosihuouusyaxtbau`, the REST host `https://tkpyosihuouusyaxtbau.supabase.co/rest/v1/`, real inference, database event streaming, and structured readback while serializing no secret values, protected source, raw prompts, provider responses, unpaid AssetPack source, wallet private material, private settlement payloads, or live log payloads.
161
161
Gate 9 permits no missing critical rehearsal automation row and keeps value-bearing mainnet admission blocked.
162
162
163
+
## V40 Gate 10 Prompt Benchmark Smoke And V41 Readiness
The package-owned `V40PromptBenchmarkSmokeV41Readiness` emits `.bitcode/v40-prompt-benchmark-smoke-v41-readiness.json` with ten source-safe rows covering the prompt benchmark report command, deterministic PromptPart smoke execution, deterministic composed Prompt smoke execution, V38 benchmark inventory binding, Reading prompt fixtures, Conversation and tool prompt fixtures, the no-prompt-rewrite boundary, the V41 prompt-program worklist, workflow wiring, and proof-system wiring.
167
+
The operator command `prompt-benchmark:smoke` runs a local deterministic mock benchmark receipt and confirms the package benchmark report command remains runnable.
168
+
Gate 10 serializes no raw prompt text, protected PromptPart source, raw provider response, credential, wallet private material, private settlement payload, or unpaid AssetPack source.
169
+
Gate 10 intentionally proves benchmark infrastructure readiness only; V41 owns every raw PromptPart and composed Prompt audit, semantic repartition, retitle, rewrite, catalogue, interpolation binding review, and callsite-quality benchmark.
170
+
163
171
## V40 canonical subsystem surfaces
164
172
165
173
### Depositing and asset supply
@@ -249,7 +257,7 @@ Current accepted boundaries: proof artifacts may summarize sensitive facts but c
V40 generated artifacts begin with `.bitcode/v40-spec-family-report.json` and `.bitcode/v40-canonical-input-report.json`.
434
-
Closed gates now add unit, API, pipeline, Conversation/Terminal, browser, synchronization, and local/staging rehearsalartifacts; later gates add prompt benchmark smoke and promotion-readiness artifacts.
443
+
Closed gates now add unit, API, pipeline, Conversation/Terminal, browser, synchronization, local/staging rehearsal, and prompt benchmark smoke artifacts; later gates add promotion-readiness artifacts.
Validation canon includes `pnpm run check:v40-gate1`, `pnpm run check:v40-gate2`, `pnpm run check:v40-gate3`, `pnpm run check:v40-gate4`, `pnpm run check:v40-gate5`, `pnpm run check:v40-gate6`, `pnpm run check:v40-gate7`, `pnpm run check:v40-gate8`, `pnpm run check:v40-gate9`, `pnpm run check:v40-browser-e2e-visual-proof`, `pnpm run rehearse:v40-local-staging --lane local --dry-run --json`, `pnpm run rehearse:v40-local-staging --lane staging-testnet --dry-run --json`, `node scripts/check-bitcode-spec-family.mjs --version V40 --mode draft --current-target V39`, `node scripts/check-bitcode-canon-posture-drift.mjs --active-canon V39 --draft-target V40`, package typechecks, protocol tests, route/API tests, browser E2E suites, visual/screenshot proof suites, integration suites, unit suites, prompt benchmark smoke tests, local/staging rehearsal commands, and diff hygiene.
469
+
Validation canon includes `pnpm run check:v40-gate1`, `pnpm run check:v40-gate2`, `pnpm run check:v40-gate3`, `pnpm run check:v40-gate4`, `pnpm run check:v40-gate5`, `pnpm run check:v40-gate6`, `pnpm run check:v40-gate7`, `pnpm run check:v40-gate8`, `pnpm run check:v40-gate9`, `pnpm run check:v40-gate10`, `pnpm run check:v40-browser-e2e-visual-proof`, `pnpm run prompt-benchmark:smoke`, `pnpm run generate:v40-prompt-benchmark-smoke-v41-readiness`, `pnpm run check:v40-prompt-benchmark-smoke-v41-readiness`, `pnpm run rehearse:v40-local-staging --lane local --dry-run --json`, `pnpm run rehearse:v40-local-staging --lane staging-testnet --dry-run --json`, `node scripts/check-bitcode-spec-family.mjs --version V40 --mode draft --current-target V39`, `node scripts/check-bitcode-canon-posture-drift.mjs --active-canon V39 --draft-target V40`, package typechecks, protocol tests, route/API tests, browser E2E suites, visual/screenshot proof suites, integration suites, unit suites, prompt benchmark smoke tests, local/staging rehearsal commands, and diff hygiene.
Gate 10 does not rewrite PromptPart or Prompt content; it proves benchmark infrastructure readiness and hands every semantic prompt-program improvement to V41.
- Source parity state: V40 testing gates are closing progressively through generated artifacts
12
12
13
13
## Notes companion rule
@@ -122,3 +122,10 @@ Its generated `.bitcode/v40-local-staging-rehearsal-automation.json` artifact bi
122
122
The operator command `rehearse:v40-local-staging` emits receipts from environment presence only; secret values, protected source, raw prompts, raw provider responses, unpaid AssetPack source, wallet private material, private settlement payloads, and live log payloads are absent from the generated artifact and receipts.
123
123
Live execution remains dry-run by default and requires `BITCODE_V40_REHEARSAL_EXECUTE=1` before delegating to the Vercel Sandbox AssetPack harness.
124
124
Staging-testnet receipts bind Supabase project `tkpyosihuouusyaxtbau`, REST host `https://tkpyosihuouusyaxtbau.supabase.co/rest/v1/`, real inference, and database event streaming/readback.
125
+
126
+
## Gate 10 implementation notes
127
+
128
+
Gate 10 is closed by `V40PromptBenchmarkSmokeV41Readiness`.
129
+
Its generated `.bitcode/v40-prompt-benchmark-smoke-v41-readiness.json` artifact binds prompt benchmark smoke coverage to ten source-safe rows: benchmark report command execution, deterministic PromptPart smoke execution, deterministic composed Prompt smoke execution, V38 benchmark inventory binding, Reading prompt fixtures, Conversation and tool prompt fixtures, source-safe no-rewrite boundary, V41 prompt-program worklist, workflow wiring, and proof-system wiring.
130
+
The `prompt-benchmark:smoke` command uses a local source-safe mock provider and records benchmark scores, fixture roots, and package benchmark report counts without serializing raw prompt text, protected PromptPart source, raw provider responses, credentials, wallet private material, private settlement payloads, or unpaid AssetPack source.
131
+
Gate 10 exists to make prompt and PromptPart benchmarking runnable and auditable before V41; every prompt semantic repartition, retitle, rewrite, catalogue improvement, interpolation binding review, and callsite-quality benchmark remains V41 work.
- Source parity state: V40 testing parity is closing progressively through generated artifacts
12
12
13
13
## Purpose
14
14
@@ -36,7 +36,7 @@ This matrix records the V40 testing surfaces that must become promotion-grade be
36
36
| Pipeline integration | Primitive and real Reading pipeline implementations are tested |`.bitcode/v40-reading-pipeline-integration-coverage.json`| implemented |
37
37
| Conversation/Terminal integration | Conversation handoff, stream logs, Terminal Reading state, and authority boundaries are tested source-safely |`.bitcode/v40-conversation-terminal-integration.json`| implemented |
38
38
| Unit coverage | Packages, primitives, isolated implementations, and real implementations have unit coverage |`.bitcode/v40-unit-coverage-inventory.json`| implemented |
39
-
| Prompt benchmark smoke | Prompt and PromptPart benchmarks run before V41 |future V40 artifacts | not yet implemented |
39
+
| Prompt benchmark smoke | Prompt and PromptPart benchmarks run before V41 |`.bitcode/v40-prompt-benchmark-smoke-v41-readiness.json` and `V40PromptBenchmarkSmokeV41Readiness`| implemented |
40
40
41
41
## V40 implementation checklist
42
42
@@ -51,7 +51,7 @@ This matrix records the V40 testing surfaces that must become promotion-grade be
| Gate 11 | Promotion readiness artifact and workflow | not yet implemented |
56
56
57
57
Gate 2 implementation evidence: package-backed `V40TestInventoryCoverageMatrix` emits `.bitcode/v40-test-inventory-coverage-matrix.json` and is wired through `check:v40-gate2`.
Gate 7 implementation evidence: package-backed `V40BrowserE2eVisualProof` emits `.bitcode/v40-browser-e2e-visual-proof.json` and is wired through `check:v40-gate7`.
63
63
Gate 8 implementation evidence: package-backed `V40LedgerStorageSync` emits `.bitcode/v40-ledger-storage-sync.json` and is wired through `check:v40-gate8`.
64
64
Gate 9 implementation evidence: package-backed `V40LocalStagingRehearsalAutomation` emits `.bitcode/v40-local-staging-rehearsal-automation.json` and is wired through `check:v40-gate9`.
65
+
Gate 10 implementation evidence: package-backed `V40PromptBenchmarkSmokeV41Readiness` emits `.bitcode/v40-prompt-benchmark-smoke-v41-readiness.json`, runs source-safe PromptPart and composed Prompt smoke receipts through `prompt-benchmark:smoke`, binds V38 benchmark inventory evidence, and is wired through `check:v40-gate10`.
0 commit comments