Skip to content

Commit a30c388

Browse files
V48 Gate 3 (specification-implementation): withhold raw llm/response in synthesis telemetry; fix log x-overflow (F18)
During /deposit "Synthesize options", the Synthesis run telemetry leaked the raw model response line-by-line (```json / { / "analysis": ... / "steps": [ each a row) and the long unwrapped lines x-overflowed the whole page. Root cause (source-safety): the formal Thricified substeps store LLM content under llm/input|prompt|output|parsedOutput, but AgentLLMsRegistry/PipelineLLMRegistry (direct getLLM calls, used by the setup-plan agent) store the raw prompt under llm/messages and the raw response under llm/response (output.content). sourceSafeStreamEvent withheld only the substep key names, so llm/response passed through as a status-event message into execution_events unredacted — and the renderer (splits output on \n) fragmented that multi-line payload into one row per line. Fix, three layers: - pipeline-stream-integration.ts: sourceSafeStreamEvent now withholds by metadata ALLOWLIST — every llm store is content-withheld except a fixed source-safe set (startTime/endTime/duration/usage/status/provider/model/configKey/stopReason/error). Robust to content-key drift across the two LLM-call paths; llm/response + llm/messages now withheld (message -> [content withheld - source-safe], data -> structural summary). Regression test added (5/5 pass). - terminal-run-activity.ts: every event line collapsed to a single bounded line (toSafeSingleLine, 280 chars) so one streamed event = exactly one accordion row and the outputDetails key lookup stays intact (defense-in-depth). - pipeline-execution-log.tsx + DepositPageClient.tsx: min-w-0 on the compact/desktop title spans and the telemetry panel section/wrapper (+ overflow-hidden on the panel) so a long line truncates within its row instead of x-overflowing the page. Spec (BITCODE_SPEC_V48_NOTES.md) records the allowlist gate + single-line/no-overflow telemetry invariant; QA ledger records F18 (incl. the read-lens setup-plan-agent-in-deposit observation queued separately). pipelines-generics tsc + uapi tsc 0; pipeline-stream-integration, terminalTransactionActivity, depositPageClient, conversation/reading pipeline-log tests green; spec checker green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent 6a13d5b commit a30c388

7 files changed

Lines changed: 132 additions & 24 deletions

File tree

BITCODE_SPEC_V48_NOTES.md

Lines changed: 19 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -182,10 +182,25 @@ Accepted V48 architecture law (decided 2026-06-25):
182182
`POST /api/deposit/synthesize-options` against the streaming execution, so the
183183
SDIVF telemetry renders end to end; the bounded-structured-inference path
184184
remains a resilience fallback when the pipeline yields no admissible options.
185-
- Source-safety is enforced universally: a content-withholding streaming filter
186-
withholds every pipeline's `llm` content events (prompt/input/output/reasoning/
187-
judgment/parsed), plus a synthesis-local defense-in-depth candidate assertion;
188-
`rawPromptVisible` / `rawProviderResponseVisible` stay false by law.
185+
- Source-safety is enforced universally by `sourceSafeStreamEvent`, the single
186+
content-withholding gate every persisted/streamed pipeline event passes through.
187+
It withholds by **metadata allowlist**, not a content-key denylist: every `llm`
188+
store is content-withheld (message → `[content withheld — source-safe]`, data →
189+
a structural summary of {stage, generation, provider, model, contentChars,
190+
phase, agent, step}) EXCEPT a fixed source-safe metadata set
191+
(`startTime/endTime/duration/usage/status/provider/model/configKey/stopReason/
192+
error`). The allowlist is required because the two LLM-call paths name their
193+
content keys differently — the Thricified substeps use `input/prompt/output/
194+
parsedOutput`, while `AgentLLMsRegistry`/`PipelineLLMRegistry` (direct `getLLM`)
195+
use `messages/config/response` — and a denylist silently leaked `llm/response`
196+
(the raw provider response) as a status-event message (QA F18). A
197+
synthesis-local defense-in-depth candidate assertion remains; `rawPromptVisible`
198+
/ `rawProviderResponseVisible` stay false by law.
199+
- Telemetry rendering is single-line and bounded: the activity builder collapses
200+
every event to exactly one bounded line (one streamed event = one accordion
201+
row, never fragmented by embedded newlines) and the log row title spans carry
202+
`min-w-0` so a long line truncates within its row rather than x-overflowing the
203+
page (QA F18).
189204
- **OTF (on-the-fly instructions) is eradicated entirely** as legacy residue:
190205
the dead telemetry types (`otf_instructions`/`otf_adherence`), the unused
191206
`setOnTheFly` prompt primitive, the live instruction-injection feature (the

BITCODE_V48_QA.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -178,6 +178,18 @@ Track 3-4 scripts (BTD ledger, settlement, pack journaling) get added when those
178178
- Cause: the option `<article>` (a grid item in the `xl:grid-cols-3` options grid) lacked `min-w-0`, and the `font-mono` covered-source-paths list lacked `break-all`, so a card with long unbreakable mono content refused to shrink below its content's min-content width and overflowed the `minmax(0,1.45fr)` left column.
179179
- Repair (2026-06-26, `DepositPageClient.tsx`): `min-w-0` on the option card + `break-all` on the covered-paths list (the Option-roots `<dd>` already wrapped). The card now shrinks to its grid column and long paths/roots wrap rather than forcing overflow.
180180

181+
### F18 — Synthesis run telemetry leaks the raw model response and x-overflows the page (one accordion row per content line)
182+
183+
- Severity: high (source-safety law) + low (layout). Telemetry must never expose raw prompts/responses (`rawProviderResponseVisible=false`).
184+
- Observed (2026-06-26): during `/deposit` "Synthesize options", the Synthesis run telemetry accordion rendered the raw model output line-by-line — ` ```json `, `{`, `"analysis": "…"`, `"steps": [`, `"1. ANCHOR IDENTITY: …"` each became their own row — and the long unwrapped `"analysis"`/step lines x-overflowed the entire page (page shifted right, panel toggle clipped). The raw content was the setup-plan agent's plan prose (a provider response).
185+
- Cause (root, source-safety): the formal Thricified substeps store LLM content under `llm/input|prompt|output|parsedOutput`, but `AgentLLMsRegistry`/`PipelineLLMRegistry` (direct `getLLM` calls, used by the setup-plan agent) store the raw prompt under `llm/messages` and the raw response under **`llm/response`** (`output.content`). The universal streaming filter `sourceSafeStreamEvent` withheld only the substep key names, so `llm/response` (a raw string) passed through as a `status`-event `message` and reached `execution_events` unredacted.
186+
- Cause (display): `buildTerminalRunActivityFromEvents` joins event messages with `\n` and `PipelineExecutionLog` splits `output` on `\n` (one row per line), so the multi-line leak fragmented into many rows; the compact/desktop title spans lacked `min-w-0`, so a long line's min-content width escaped the `overflow-auto` log container and widened the page.
187+
- Repair (2026-06-26):
188+
- `pipeline-stream-integration.ts``sourceSafeStreamEvent` now withholds by **metadata allowlist**: every `llm` store is content-withheld except a fixed source-safe set (`startTime/endTime/duration/usage/status/provider/model/configKey/stopReason/error`). Robust to content-key drift between the two LLM-call paths; `llm/response` + `llm/messages` are now withheld (message → `[content withheld — source-safe]`, `data` → structural summary). Regression test added.
189+
- `terminal-run-activity.ts` — every event line is collapsed to a single bounded line (`toSafeSingleLine`, 280 chars) so one event = exactly one row and the `outputDetails` key lookup stays intact (defense-in-depth).
190+
- `pipeline-execution-log.tsx` + `DepositPageClient.tsx``min-w-0` on the compact/desktop title spans and on the telemetry panel section/wrapper (+ `overflow-hidden` on the panel) so a long line truncates within its row instead of x-overflowing the page.
191+
- Related observation (not yet filed): the deposit run executes `ReadFitsFindingSynthesisSetupPlanAgent` planning a "Read-Need" — a read-lens setup agent running under the deposit lens. Telemetry is now source-safe regardless, but the deposit setup-plan agent identity should be confirmed/queued as a separate correctness finding.
192+
181193
## Track 1 — Identity / Authentication / Auxillaries — COMPLETE 2026-06-12 (email deferred by decision; F2/F9 and legacy eradication queued for gates)
182194

183195
- [x] Sign up / sign in via Connect Wallet (nav CTA → SignUpWindow → wallet signature on testnet4 → `custom:bitcode-bitcoin` session → `/tps/supabase/callback`) — verified 2026-06-12 after F5 fix; lands on `/packs`. Re-verified from fully nuked state (purged user + cleared site data): created 19:29:21 → session 19:29:25 → binding auto-written 19:29:29 by the bridge on `/packs` mount with no Auxillaries visit; UI consistent across nav, Wallet, and Profile panes.

packages/pipelines-generics/src/streaming/__tests__/pipeline-stream-integration.test.ts

Lines changed: 48 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
// @ts-nocheck
22
import { Execution } from '@bitcode/execution-generics';
3-
import { enablePipelineStreaming } from '../../streaming/pipeline-stream-integration';
3+
import { enablePipelineStreaming, sourceSafeStreamEvent } from '../../streaming/pipeline-stream-integration';
44

55
// Mock ORM model so we can assert persistence without a real DB
66
const createdEvents: any[] = [];
@@ -40,3 +40,50 @@ describe('pipeline-stream-integration', () => {
4040
expect(createdEvents[0].event_type).toBeDefined();
4141
});
4242
});
43+
44+
describe('sourceSafeStreamEvent (telemetry source-safety law, V48)', () => {
45+
const RAW_RESPONSE = '```json\n{\n "analysis": "secret plan prose",\n "steps": ["step one"]\n}\n```';
46+
47+
it('withholds the raw model response stored under llm/response', () => {
48+
const safe = sourceSafeStreamEvent({
49+
type: 'status',
50+
namespace: 'llm',
51+
key: 'response',
52+
message: RAW_RESPONSE,
53+
data: RAW_RESPONSE,
54+
executionState: { phase: 'setup', agent: 'SetupPlanAgent', step: 'plan' },
55+
});
56+
expect(safe.message).toBe('[content withheld — source-safe]');
57+
expect(safe.data.contentWithheld).toBe(true);
58+
expect(safe.data.sourceSafetyClass).toBe('source_safe');
59+
expect(safe.data.contentChars).toBe(RAW_RESPONSE.length);
60+
// The raw prose must not survive anywhere on the event.
61+
expect(JSON.stringify(safe)).not.toContain('secret plan prose');
62+
});
63+
64+
it('withholds the raw prompt stored under llm/messages', () => {
65+
const safe = sourceSafeStreamEvent({
66+
type: 'status',
67+
namespace: 'llm',
68+
key: 'messages',
69+
data: [{ role: 'user', content: 'secret prompt body' }],
70+
});
71+
expect(safe.message).toBe('[content withheld — source-safe]');
72+
expect(JSON.stringify(safe)).not.toContain('secret prompt body');
73+
});
74+
75+
it('passes through source-safe llm metadata (usage) unchanged', () => {
76+
const event = {
77+
type: 'status',
78+
namespace: 'llm',
79+
key: 'usage',
80+
data: { promptTokens: 10, completionTokens: 20 },
81+
};
82+
expect(sourceSafeStreamEvent(event)).toBe(event);
83+
});
84+
85+
it('passes through non-llm events unchanged', () => {
86+
const event = { type: 'phase-start', namespace: 'phase', key: 'start', message: 'Setup phase started' };
87+
expect(sourceSafeStreamEvent(event)).toBe(event);
88+
});
89+
});

packages/pipelines-generics/src/streaming/pipeline-stream-integration.ts

Lines changed: 26 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -27,25 +27,38 @@ export interface PipelineStreamConfig {
2727
}
2828

2929
// Source-safety law (V48): pipeline telemetry must never serialize raw prompts
30-
// or provider responses (rawPromptVisible/rawProviderResponseVisible=false). The
31-
// formal LLM substeps store full prompt/response content under the `llm`
32-
// namespace, which auto-streams via Execution.store. This withholds that content
33-
// from any persisted/streamed event, leaving only a content-withheld summary —
34-
// applied universally to every pipeline stream event (not per-pipeline).
35-
const SOURCE_SAFE_LLM_CONTENT_KEYS = new Set([
36-
'prompt',
37-
'input',
38-
'output',
39-
'reasoningOutput',
40-
'judgmentOutput',
41-
'parsedOutput',
30+
// or provider responses (rawPromptVisible/rawProviderResponseVisible=false). Raw
31+
// prompt/response content auto-streams via Execution.store under the `llm`
32+
// namespace, but the content-bearing key NAMES drift between the two LLM-call
33+
// paths: the formal Thricified substeps store `input`/`prompt`/`output`/
34+
// `parsedOutput`, while AgentLLMsRegistry/PipelineLLMRegistry (direct getLLM
35+
// calls) store `messages`/`config`/`response`. A content-key denylist silently
36+
// missed `response`, so a raw model response leaked through as a status-event
37+
// message (and the renderer split that multi-line payload into one row per
38+
// line). We therefore withhold by ALLOWLIST: every `llm` store is content-
39+
// withheld EXCEPT a fixed set of source-safe metadata keys. This is robust to
40+
// new content keys and is applied universally to every pipeline stream event
41+
// (not per-pipeline).
42+
const SOURCE_SAFE_LLM_METADATA_KEYS = new Set([
43+
'startTime',
44+
'endTime',
45+
'duration',
46+
'usage',
47+
'status',
48+
'provider',
49+
'model',
50+
'configKey',
51+
'stopReason',
52+
'error',
4253
]);
4354

4455
export function sourceSafeStreamEvent(event: any): any {
4556
if (!event || typeof event !== 'object') return event;
4657
const namespace = (event as any).namespace;
4758
const key = (event as any).key;
48-
if (namespace !== 'llm' || !SOURCE_SAFE_LLM_CONTENT_KEYS.has(String(key))) {
59+
// Only llm-namespace stores carry raw prompt/response content. Anything that
60+
// is an llm metadata store, or any non-llm event, passes through unchanged.
61+
if (namespace !== 'llm' || SOURCE_SAFE_LLM_METADATA_KEYS.has(String(key))) {
4962
return event;
5063
}
5164
const data =

uapi/app/deposit/DepositPageClient.tsx

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1300,7 +1300,7 @@ export default function DepositPageClient() {
13001300
{synthesisRunId ? (
13011301
<section
13021302
ref={synthesisTelemetryRef}
1303-
className="border border-white/10 bg-white/[0.035] px-4 py-4"
1303+
className="min-w-0 overflow-hidden border border-white/10 bg-white/[0.035] px-4 py-4"
13041304
aria-label="AssetPacksSynthesis run telemetry"
13051305
data-testid="deposit-synthesis-telemetry"
13061306
>
@@ -1323,7 +1323,7 @@ export default function DepositPageClient() {
13231323
{synthesisRunId}
13241324
</span>
13251325
</div>
1326-
<div className="mt-4">
1326+
<div className="mt-4 min-w-0">
13271327
<PipelineExecutionLog
13281328
output={synthesisActivity.output}
13291329
outputDetails={synthesisActivity.outputDetails}

uapi/app/terminal/terminal-run-activity.ts

Lines changed: 23 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,25 @@ export type MockRunActivitySnapshot = {
3535
error?: string | null;
3636
};
3737

38+
// One streamed event must render as exactly one accordion row. The renderer
39+
// (PipelineExecutionLog) splits `output` on '\n', so any embedded newline in an
40+
// event message would fragment a single event into many rows (and break the
41+
// outputDetails key lookup, since the key is the full multi-line string). We
42+
// therefore collapse every event line to a single bounded line. Raw model
43+
// content is already withheld upstream by sourceSafeStreamEvent; this is the
44+
// client-side guarantee that nothing can fragment or overflow the log even if a
45+
// future event slips a newline through.
46+
const MAX_ACTIVITY_LINE_CHARS = 280;
47+
function toSafeSingleLine(value: string): string {
48+
const collapsed = String(value ?? '')
49+
.replace(/\s*\r?\n\s*/g, ' ')
50+
.replace(/\s{2,}/g, ' ')
51+
.trim();
52+
return collapsed.length > MAX_ACTIVITY_LINE_CHARS
53+
? `${collapsed.slice(0, MAX_ACTIVITY_LINE_CHARS - 1)}…`
54+
: collapsed;
55+
}
56+
3857
function normalizeEventMessage(payload: any) {
3958
if (!payload || typeof payload !== 'object') return null;
4059

@@ -102,8 +121,10 @@ export function buildTerminalRunActivityFromEvents(
102121

103122
const normalized = normalizeEventMessage(payload);
104123
if (!normalized) continue;
105-
outputLines.push(normalized.text);
106-
outputDetails[normalized.text] = payload;
124+
const safeText = toSafeSingleLine(normalized.text);
125+
if (!safeText) continue;
126+
outputLines.push(safeText);
127+
outputDetails[safeText] = payload;
107128
}
108129

109130
const latestStatusEvent = statusEvents[statusEvents.length - 1];

uapi/components/base/bitcode/execution/pipeline-execution-log.tsx

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -869,7 +869,7 @@ function renderLogLine(
869869
/>
870870
<span
871871
title={logLine.text}
872-
className="truncate flex-1 text-[0.82rem] leading-none m-0"
872+
className="truncate flex-1 min-w-0 text-[0.82rem] leading-none m-0"
873873
>
874874
{logLine.text}
875875
</span>
@@ -995,7 +995,7 @@ function renderLogLine(
995995
{/* Main text */}
996996
<span
997997
title={logLine.text}
998-
className="select-text cursor-text truncate pr-3 text-xs tablet:text-sm laptop:text-[0.94rem] desktop:text-base font-medium leading-none h-5 flex items-center gap-1"
998+
className="select-text cursor-text truncate min-w-0 flex-1 pr-3 text-xs tablet:text-sm laptop:text-[0.94rem] desktop:text-base font-medium leading-none h-5 flex items-center gap-1"
999999
>
10001000
<Icon className="inline-block laptop:hidden w-4 h-4 text-current" />
10011001
{logLine.text}

0 commit comments

Comments
 (0)