Skip to content

Commit 36d91bb

Browse files
wip v28
1 parent e6abc90 commit 36d91bb

13 files changed

Lines changed: 1180 additions & 608 deletions

BITCODE_SPEC_V28.md

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -385,7 +385,7 @@ Acceptance criteria:
385385

386386
#### Pipeline host runtime environment for Read/Fit QA
387387

388-
V28 now admits a lightweight deployment harness because the first commercial
388+
V28 now admits a lightweight deployment harness because the first value-bearing
389389
Read/Fit result cannot be reviewed from local posture rows alone.
390390
The harness is not the full V34 distributed deployment program; it is the
391391
minimum real host contract needed to prove that AssetPack pipeline execution,
@@ -512,6 +512,17 @@ Read/Fit result review remains fail-closed:
512512
still needs a server-side model credential such as `OPENAI_API_KEY`, real
513513
Supabase admin/service-role credentials for telemetry persistence, and a
514514
runtime budget that is shorter than the enclosing Vercel Function window.
515+
- a local application deployment may stand in for route/UI implementation QA
516+
when live deployment is explicitly out of scope. In that mode the operator
517+
must run the Terminal application locally with
518+
`BITCODE_PIPELINE_HARNESS_REQUIRE_REAL_INFERENCE=1`,
519+
`BITCODE_ASSET_PACK_REAL_INFERENCE=1`,
520+
`BITCODE_ASSET_PACK_REAL_INFERENCE_PROFILE=bounded`, aligned Supabase
521+
service-role credentials, OpenAI credentials, and Vercel Sandbox local auth.
522+
Local application deployment evidence can prove route preflight, canonical
523+
stream rendering, artifact streaming, and SQL readback behavior, but it
524+
cannot close a live source-revision settlement/finality gate unless the same
525+
source and database/ledger rows are read back from the accepted environment.
515526

516527
### Gate 4: Terminal AssetPack Range Detail
517528

BITCODE_V28_QA.md

Lines changed: 38 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1518,14 +1518,50 @@ manifest-bound Deposit evidence root fixes:
15181518
present, DB URL counting, optional `phase_executions` substitution, and
15191519
ledger-row blockers.
15201520
1521+
2026-05-18 local-only implementation pass:
1522+
1523+
- No live deployment is part of this pass. V28 route, stream, and harness work
1524+
is validated through local application deployment only.
1525+
- Local Terminal application deployment must start with
1526+
`BITCODE_ENABLE_PIPELINE_HARNESS_API=1`,
1527+
`BITCODE_PIPELINE_HARNESS_REQUIRE_REAL_INFERENCE=1`,
1528+
`BITCODE_ASSET_PACK_REAL_INFERENCE=1`,
1529+
`BITCODE_ASSET_PACK_REAL_INFERENCE_PROFILE=bounded`, and
1530+
`BITCODE_PIPELINE_HARNESS_MAX_RUNTIME_MS<=600000` so local `next dev`
1531+
cannot silently run deterministic Read/Fit branches.
1532+
- Local strict route preflight now reports whether real inference is required,
1533+
whether OpenAI and Supabase service-role credentials are present, the
1534+
inference profile, the runtime budget, and the database host before sandbox
1535+
creation. The canonical Terminal execution stream panel remains the operator
1536+
surface for sandbox id, run id, phase, agent, generation/tool, and parsed
1537+
output metadata.
1538+
- The pipeline harness route now delegates execution to a route runner boundary,
1539+
keeping the Next route responsible only for auth, validation, and SSE framing.
1540+
Focused local tests call the same runner directly, proving strict preflight
1541+
failure before sandbox creation, strict success event order, command
1542+
environment forwarding, summarized evidence shape, and secret redaction.
1543+
The same test file also covers wrapper fail-closed behavior for production
1544+
disablement, missing auth, and incomplete request bodies before sandbox work.
1545+
- Local application deployment evidence can close route wiring, UI streaming,
1546+
preflight, and readback-verifier acceptance criteria. It cannot close the
1547+
clean source-revision settlement/finality gate while live deployment is
1548+
intentionally out of scope; that gate still requires BTD range, BTC fee,
1549+
ownership/license, Terminal journal, ledger anchor, and crypto telemetry rows
1550+
to be written and read back in the accepted environment.
1551+
- Re-running the readback verifier on 2026-05-18 with `.env.local`,
1552+
`--readback-source db`, and a 48 hour lookback remains correctly blocked:
1553+
one pipeline run and deliverable stream telemetry are visible, but the REST
1554+
Supabase host still differs from the DB host and settlement rows remain zero.
1555+
15211556
Pass 2C prompt-to-artifact closure audit:
15221557
15231558
| Requirement | Current artifact/evidence | Gate state |
15241559
| --- | --- | --- |
15251560
| Vercel Sandbox harness creates a host, runs commands, exports evidence, exports telemetry, and cleans up. | `packages/pipeline-hosts` host/manifest/harness tests pass; local sandbox artifacts were exported for `sbx_rLVfPTD3HuITtCbrR0AmZ26spEYO`. | implemented and locally verified |
1526-
| Route-started run id is visible before telemetry starts. | Deployed route code allocates `BITCODE_PIPELINE_RUN_ID`; Terminal stream snapshot and metadata rows consume `runId`; focused UAPI tests cover the stream adapter. | implemented and tested |
1561+
| Route-started run id is visible before telemetry starts. | The route runner allocates `BITCODE_PIPELINE_RUN_ID`; Terminal stream snapshot and metadata rows consume `runId`; focused UAPI tests cover the stream adapter and runner event order. | implemented and tested |
15271562
| Staging route uses real bounded inference, not deterministic bring-up or full-profile blocking. | Route preflight requires real inference, OpenAI key, `bounded`, and runtime budget `<=600000`; Terminal summarizes full-profile async blocker. | implemented and tested |
1528-
| Operators can see sanitized preflight context before waiting on the sandbox. | SSE preflight includes Supabase host, profile, and runtime budget; Terminal stream metadata renders database/profile/budget. | implemented and tested |
1563+
| Local application deployment can enforce the same route strictness without deploying. | `BITCODE_PIPELINE_HARNESS_REQUIRE_REAL_INFERENCE=1` makes local `next dev` require real bounded inference, OpenAI, aligned Supabase service-role credentials, and route budget `<=600000`. `uapi/tests/api/pipelineHarnessRoute.test.ts` proves local strict failure and strict success without live deployment. | implemented and tested |
1564+
| Operators can see sanitized preflight context before waiting on the sandbox. | SSE preflight includes Supabase host, profile, and runtime budget; Terminal stream metadata renders database/profile/budget; route-runner tests assert secret redaction in completion tails. | implemented and tested |
15291565
| A repeatable readback verifier exists for staging operators. | `pnpm qa:v28:pipeline-readback -- --env-file .env.local --expected-host tkpyosihuouusyaxtbau.supabase.co` checks sanitized REST/DB host identity, admin credential posture, pipeline telemetry tables, generation/tool rows, and ledger settlement rows. `--readback-source db` can read staging rows even when REST env drift would otherwise hide them. `pnpm test:qa:v28:pipeline-readback` passes 8 verifier tests. | implemented and tested; current readback is blocked by mixed REST/DB project env and missing settlement rows |
15301566
| Deposited proof/measurement flags become manifest-bound roots before Fit evaluation. | Harness materializes deterministic proof, measurement, and reconciliation roots; focused harness test asserts root shapes. | implemented and tested |
15311567
| Search/finding produces query root, ranking root, selected candidate ids, and embedding policy. | Local overlay run selected `manual-deposit-qa` and recorded query/ranking roots with OpenAI `text-embedding-3-small`. | locally verified only |

packages/pipeline-hosts/README.md

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -93,6 +93,14 @@ rejects `full` preflight.
9393
The deployed route assigns `BITCODE_PIPELINE_RUN_ID` before sandbox creation and
9494
echoes that id through each SSE event so Terminal can display a stable run id
9595
before telemetry artifacts are written.
96+
Local application deployments use the same route, stream, and harness code
97+
without deploying to Vercel. To make local route QA as strict as staging, set
98+
`BITCODE_PIPELINE_HARNESS_REQUIRE_REAL_INFERENCE=1` alongside
99+
`BITCODE_ASSET_PACK_REAL_INFERENCE=1`,
100+
`BITCODE_ASSET_PACK_REAL_INFERENCE_PROFILE=bounded`, OpenAI credentials,
101+
aligned Supabase service-role credentials, and Vercel Sandbox local auth.
102+
Without the explicit local strict flag, ordinary `next dev` may inspect UI and
103+
route wiring, but it is not admissible Read/Fit closure evidence.
96104
After a clean deployed run, verify the staging database and ledger readback with
97105
`pnpm qa:v28:pipeline-readback -- --env-file .env.local --expected-host <staging-supabase-host>`.
98106
The verifier emits sanitized counts and V28 blockers without printing secrets.
@@ -125,6 +133,27 @@ larger harness runtime budget.
125133

126134
## Live QA
127135

136+
### Local application deployment
137+
138+
For the current no-deploy pass, start the Terminal locally and keep the
139+
real-inference route gate enabled:
140+
141+
```bash
142+
BITCODE_ENABLE_PIPELINE_HARNESS_API=1 \
143+
BITCODE_PIPELINE_HARNESS_REQUIRE_REAL_INFERENCE=1 \
144+
BITCODE_ASSET_PACK_REAL_INFERENCE=1 \
145+
BITCODE_ASSET_PACK_REAL_INFERENCE_PROFILE=bounded \
146+
BITCODE_PIPELINE_HARNESS_MAX_RUNTIME_MS=600000 \
147+
pnpm -C uapi dev:remote
148+
```
149+
150+
The local process must have `OPENAI_API_KEY`, aligned Supabase URL/service-role
151+
credentials, and either `VERCEL_OIDC_TOKEN` from `vercel link && vercel env pull`
152+
or the access-token tuple used by `@vercel/sandbox`. This command is a local
153+
application deployment only; it must not be counted as clean live settlement
154+
evidence until the resulting database and ledger rows pass SQL readback for the
155+
accepted environment.
156+
128157
Link a Vercel project and pull local OIDC credentials:
129158

130159
```bash
Lines changed: 154 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,154 @@
1+
export type PipelineHarnessPreflightBody = {
2+
repositoryFullName?: string;
3+
sourceBranch?: string;
4+
sourceCommit?: string;
5+
readId?: string;
6+
depositId?: string;
7+
};
8+
9+
type PipelineHarnessRuntimeEnv = Record<string, string | undefined>;
10+
11+
export function isProductionDeployment(env: PipelineHarnessRuntimeEnv = process.env): boolean {
12+
return env.VERCEL_ENV === 'production';
13+
}
14+
15+
export function isDeployedRuntime(env: PipelineHarnessRuntimeEnv = process.env): boolean {
16+
return env.VERCEL === '1' || env.NODE_ENV === 'production';
17+
}
18+
19+
export function isEnabled(value: string | undefined): boolean {
20+
return ['1', 'true', 'yes', 'on'].includes(String(value || '').trim().toLowerCase());
21+
}
22+
23+
export function isPipelineHarnessRealInferenceRequired(env: PipelineHarnessRuntimeEnv = process.env): boolean {
24+
return (
25+
isDeployedRuntime(env) ||
26+
isEnabled(env.BITCODE_PIPELINE_HARNESS_REQUIRE_REAL_INFERENCE)
27+
);
28+
}
29+
30+
export function isUsableSupabaseUrl(value: string | undefined): boolean {
31+
if (!value) return false;
32+
try {
33+
const host = new URL(value).host;
34+
return Boolean(host && host !== 'your-project.supabase.co' && !host.includes('<'));
35+
} catch {
36+
return false;
37+
}
38+
}
39+
40+
export function readSupabaseHost(value: string | undefined): string | null {
41+
if (!value) return null;
42+
try {
43+
const host = new URL(value).host;
44+
return host || null;
45+
} catch {
46+
return null;
47+
}
48+
}
49+
50+
export function isUsableSecretValue(value: string | undefined): boolean {
51+
return typeof value === 'string' && value.trim().length > 16 && !value.includes('<');
52+
}
53+
54+
export function readRealInferenceProfile(value: string | undefined): string {
55+
return String(value || '').trim().toLowerCase();
56+
}
57+
58+
export function assertDatabaseStreamingEnvironment(env: Record<string, string>): void {
59+
const url = env.SUPABASE_URL || env.NEXT_PUBLIC_SUPABASE_URL;
60+
const key = env.SUPABASE_SERVICE_ROLE_KEY || env.SUPABASE_SECRET_KEY || env.SUPABASE_ADMIN_KEY;
61+
if (!isUsableSupabaseUrl(url) || !isUsableSecretValue(key)) {
62+
throw new Error(
63+
'Pipeline harness database streaming requires a non-placeholder Supabase URL and service-role key.'
64+
);
65+
}
66+
}
67+
68+
export function assertRealInferenceEnvironment(env: Record<string, string>): void {
69+
if (!isPipelineHarnessRealInferenceRequired(env)) return;
70+
if (!isEnabled(env.BITCODE_ASSET_PACK_REAL_INFERENCE)) {
71+
throw new Error(
72+
'Staging pipeline harness requires BITCODE_ASSET_PACK_REAL_INFERENCE=1 so Read/Fit QA cannot run deterministic bring-up branches.'
73+
);
74+
}
75+
if (!env.OPENAI_API_KEY) {
76+
throw new Error(
77+
'Staging pipeline harness requires OPENAI_API_KEY for real AssetPack PTRR inference.'
78+
);
79+
}
80+
if (readRealInferenceProfile(env.BITCODE_ASSET_PACK_REAL_INFERENCE_PROFILE) !== 'bounded') {
81+
throw new Error(
82+
'The current Terminal harness route requires BITCODE_ASSET_PACK_REAL_INFERENCE_PROFILE=bounded. Full profile runs are scoped to the later async sandbox completion gate where the sandbox pushes finished state to a server-side stream handler.'
83+
);
84+
}
85+
const budgetMs = Number(env.BITCODE_PIPELINE_HARNESS_MAX_RUNTIME_MS || 240000);
86+
if (!Number.isFinite(budgetMs) || budgetMs <= 0) {
87+
throw new Error('BITCODE_PIPELINE_HARNESS_MAX_RUNTIME_MS must be a positive millisecond budget.');
88+
}
89+
if (budgetMs > 600000) {
90+
throw new Error(
91+
'BITCODE_PIPELINE_HARNESS_MAX_RUNTIME_MS must be <= 600000 when real-inference route strictness is required so the caller can still collect artifacts.'
92+
);
93+
}
94+
}
95+
96+
export function normalizeModelEnvironment(env: Record<string, string>): void {
97+
const provider = env.BITCODE_LLM_PROVIDER?.trim().toLowerCase();
98+
if (provider && !hasModelProviderCredential(provider, env)) {
99+
delete env.BITCODE_LLM_PROVIDER;
100+
delete env.BITCODE_LLM_MODEL;
101+
}
102+
103+
if (!env.BITCODE_LLM_PROVIDER && env.OPENAI_API_KEY) {
104+
env.BITCODE_LLM_PROVIDER = 'openai';
105+
}
106+
}
107+
108+
function hasModelProviderCredential(provider: string, env: Record<string, string>): boolean {
109+
switch (provider) {
110+
case 'openai':
111+
return Boolean(env.OPENAI_API_KEY);
112+
case 'anthropic':
113+
return Boolean(env.ANTHROPIC_API_KEY);
114+
case 'google':
115+
return Boolean(env.GOOGLE_GENERATIVE_AI_API_KEY || env.GEMINI_API_KEY || env.GOOGLE_API_KEY);
116+
default:
117+
return true;
118+
}
119+
}
120+
121+
export function summarizeHarnessPreflight(
122+
body: PipelineHarnessPreflightBody,
123+
env: PipelineHarnessRuntimeEnv = process.env
124+
): Record<string, unknown> {
125+
const supabaseUrl = env.SUPABASE_URL || env.NEXT_PUBLIC_SUPABASE_URL;
126+
const serviceRole =
127+
env.SUPABASE_SERVICE_ROLE_KEY ||
128+
env.SUPABASE_SECRET_KEY ||
129+
env.SUPABASE_ADMIN_KEY;
130+
const budgetMs = Number(env.BITCODE_PIPELINE_HARNESS_MAX_RUNTIME_MS || 240000);
131+
const realInferenceProfile = readRealInferenceProfile(
132+
env.BITCODE_ASSET_PACK_REAL_INFERENCE_PROFILE
133+
);
134+
const realInferenceRequired = isPipelineHarnessRealInferenceRequired(env);
135+
return {
136+
repositoryFullName: body.repositoryFullName || null,
137+
sourceBranch: body.sourceBranch || null,
138+
sourceCommit: body.sourceCommit || null,
139+
readId: body.readId || null,
140+
depositId: body.depositId || null,
141+
realInferenceEnabled: isEnabled(env.BITCODE_ASSET_PACK_REAL_INFERENCE),
142+
realInferenceRequired,
143+
openaiCredentialProvided: isUsableSecretValue(env.OPENAI_API_KEY),
144+
supabaseUrlProvided: isUsableSupabaseUrl(supabaseUrl),
145+
supabaseHost: readSupabaseHost(supabaseUrl),
146+
supabaseServiceRoleProvided: isUsableSecretValue(serviceRole),
147+
runtimeBudgetMs: Number.isFinite(budgetMs) ? budgetMs : null,
148+
realInferenceProfile: realInferenceProfile || (realInferenceRequired ? 'bounded' : null),
149+
fullProfileRequiresAsyncCompletion:
150+
realInferenceRequired &&
151+
realInferenceProfile === 'full',
152+
deployedRuntime: isDeployedRuntime(env),
153+
};
154+
}

0 commit comments

Comments
 (0)