Skip to content

Commit b09513b

Browse files
V48 Gate 3 (implementation-only): telemetry label correctness (Retry no longer renders 'Try'; 'Prepare Context'), 'Copy terse logs', call-ordered raw-LLM-IO sidecar
- Fix the 'retry'.includes('try') label-collision class: normalizeStepName (agent-generics + the operative uapi copy) now tests 'retry' before 'try'; same fix in the PTRR-snapshot step inference and the PathPill step-icon rules; the header strip normalizes its step chips. A Retry step rendering as 'Try' after Refine presented as a PTRR sequencing bug — the engine order plan→try→refine→retry is checker-pinned canon (protocol V38_PTRR_STEP_IDS) and is unchanged. Retry rows now label, tooltip (STEP_SPECIFICS.retry was unreachable), icon, gerund ('Retrying'), and selector-highlight as Retry. - Normalize the PrepareConciseContext display label to 'Prepare Context': FAILSAFE_PILL_NAMES entry, tooltip title override + generic failsafe copy, and the PTRR-snapshot raw machine-id leak now formats through formatMeta. Machine ids (prepare_concise_context et al.) unchanged. - Add 'Copy terse logs' beside 'Copy raw logs': events compact to timestamp/type/namespace/key/call-chain/usage/message/error rows (error and message fields keep a 2000-char budget; other strings 200-char previews); the outputDetails duplication is omitted. A much smaller payload that keeps ordering, hierarchy, and failure forensics. - writeRawLLMIO claims its monotonic sequence number before any await so fire-and-forget concurrent writes cannot swap on-disk file numbers (the judge-request-before-reason-response interleavings in run sidecars were logging artifacts, not engine concurrency). Verified: uapi jest 155 suites / 597 tests green; agent-generics 19 suites / 99 tests green; uapi tsc --noEmit clean; eslint clean on touched files. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent 93494cf commit b09513b

10 files changed

Lines changed: 415 additions & 37 deletions

File tree

packages/agent-generics/src/phaseHelpers/normalizeStepName.ts

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,9 +14,11 @@ export function normalizeStepName(step?: string): string {
1414
const lower = step.toLowerCase();
1515

1616
if (lower.includes('plan')) return 'Plan';
17+
// 'retry' must be tested before 'try': 'retry'.includes('try') is true, so
18+
// the broader match would relabel every Retry step as Try.
19+
if (lower.includes('retry')) return 'Retry';
1720
if (lower.includes('try')) return 'Try';
1821
if (lower.includes('refine')) return 'Refine';
19-
if (lower.includes('retry')) return 'Retry';
2022
if (lower.includes('generate')) return 'Try';
2123
if (lower.includes('intensify')) return 'Retry';
2224

packages/logger/src/logger.ts

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -265,11 +265,14 @@ export async function writeRawLLMIO(opts: {
265265
// exercises this same LLM-substep code path with mocked LLMs.
266266
if (process?.env?.NODE_ENV === 'test') return undefined;
267267

268-
const runDir = joinPath(RAW_LLM_LOG_BASE_DIR, sanitizeId(opts.executionId || 'run'));
269-
try { await fs.mkdir(runDir, { recursive: true }); } catch {}
270-
268+
// Claim the sequence number before any await: callers fire-and-forget, so
269+
// an await ahead of the increment lets concurrent writes swap numbers and
270+
// the on-disk ordering stops reflecting call order.
271271
const safe = (s: any) => String(s || '').toLowerCase().replace(/[^a-z0-9-_]+/g, '-').slice(0, 160) || 'na';
272272
const seq = String(++__rawLLMIOSeq).padStart(5, '0');
273+
274+
const runDir = joinPath(RAW_LLM_LOG_BASE_DIR, sanitizeId(opts.executionId || 'run'));
275+
try { await fs.mkdir(runDir, { recursive: true }); } catch {}
273276
const filename = joinPath(runDir, `${seq}-${safe(opts.pathKey)}.${opts.kind}.json`);
274277

275278
const payload = opts.kind === 'error'

uapi/app/deposit/deposit-explainers.ts

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -146,11 +146,12 @@ export const DEPOSIT_SECTION_EXPLAINERS = {
146146
summary:
147147
'A live, source-safe stream of the SynthesizeAssetPacks pipeline actually running — every phase, agent, step, and LLM/tool call, with prompt and response content withheld by law.',
148148
detail:
149-
'Rows appear only for completed LLM calls and tool uses, each carrying its full Phase→Agent→Step→Failsafe→Thinkings hierarchy; the processing indicator underneath shows what is currently running and how long since the last update, so a genuine stall is visible instead of an unexplained gap. Use "Copy raw logs" to grab the full run for support/debugging.',
149+
'Rows appear only for completed LLM calls and tool uses, each carrying its full Phase→Agent→Step→Failsafe→Thinkings hierarchy; the processing indicator underneath shows what is currently running and how long since the last update, so a genuine stall is visible instead of an unexplained gap. Use "Copy raw logs" to grab the full run for support/debugging, or "Copy terse logs" for a compact distillation (hierarchy, ordering, usage, and errors — long bodies truncated).',
150150
points: [
151151
'Watch which phase/agent is running in real time',
152152
'The processing indicator flags a likely stall after ~90s of silence',
153153
'Copy raw logs exports the full source-safe run for debugging',
154+
'Copy terse logs exports a compact distillation — call chains, ordering, usage, errors',
154155
],
155156
references: {
156157
source: [

uapi/components/base/bitcode/execution/PathPill.tsx

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -65,11 +65,13 @@ const PHASE_ICON_RULES: IconRule[] = [
6565

6666
const STEP_ICON_RULES: IconRule[] = [
6767
['plan', ClipboardIcon],
68+
// 'retry' must precede 'try': rules are substring-matched in order and
69+
// 'retry'.includes('try'), so the broader rule would claim Retry labels.
70+
['retry', LightningBoltIcon],
71+
['intensify', LightningBoltIcon],
6872
['try', MagicWandIcon],
6973
['generate', MagicWandIcon],
7074
['refine', MixerHorizontalIcon],
71-
['retry', LightningBoltIcon],
72-
['intensify', LightningBoltIcon],
7375
];
7476

7577
const FAILSAFE_ICON_RULES: IconRule[] = [

uapi/components/base/bitcode/execution/execution-telemetry-format.ts

Lines changed: 9 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -40,19 +40,20 @@ export function titleCaseWords(value: string): string {
4040
.join(' ');
4141
}
4242

43-
// "prepare_concise_context" -> "Prepare Concise Context"; "Discovery" -> "Discovery".
43+
// "chunk_then_sum" -> "Chunk Then Sum"; "Discovery" -> "Discovery".
4444
export function humanizeNounPhrase(value: string): string {
4545
return titleCaseWords(value.replace(/[_-]/g, ' '));
4646
}
4747

4848
// Client-side normalized failsafe names: ChunkThenSum handles LARGE INPUTS,
49-
// StitchComplete handles LARGE OUTPUTS; PrepareConciseContext keeps its
50-
// descriptive name on the pill. The log title-line (pill) reads 'handle large
49+
// StitchComplete handles LARGE OUTPUTS; PrepareConciseContext displays as
50+
// 'Prepare Context' (the 'Concise' qualifier is an internal naming detail,
51+
// dropped from the label). The log title-line (pill) reads 'handle large
5152
// inputs'; the processing sentence reads title-cased without 'handle'
5253
// ('Reasoning over Large Inputs'). PrepareConciseContext reads simply
53-
// 'Context' in the sentence ('Reasoning over Context') while the pill keeps
54-
// the full 'Prepare Concise Context' name.
54+
// 'Context' in the sentence ('Reasoning over Context').
5555
export const FAILSAFE_PILL_NAMES: Record<string, string> = {
56+
prepare_concise_context: 'Prepare Context',
5657
chunk_then_sum: 'handle large inputs',
5758
stitch_until_complete: 'handle large outputs',
5859
};
@@ -146,9 +147,11 @@ export function normalizeStepName(step: string | undefined): string {
146147
const stepLower = step.toLowerCase();
147148

148149
if (stepLower.includes('plan')) return 'Plan';
150+
// 'retry' must be tested before 'try': 'retry'.includes('try') is true, so
151+
// the broader match would relabel every Retry step as Try.
152+
if (stepLower.includes('retry')) return 'Retry';
149153
if (stepLower.includes('try')) return 'Try';
150154
if (stepLower.includes('refine')) return 'Refine';
151-
if (stepLower.includes('retry')) return 'Retry';
152155
if (stepLower.includes('generate')) return 'Try';
153156
if (stepLower.includes('intensify')) return 'Retry';
154157
if (stepLower.includes('initialize')) return 'Initialize';

uapi/components/base/bitcode/execution/pipeline-execution-log-header.tsx

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
import React from 'react';
22

33
import { ExecutionPhase, ExecutionStep, FailsafeStep, GenerationStep } from '@bitcode/streams';
4+
import { normalizeStepName } from './execution-telemetry-format';
45
import styles from './pipeline-execution-log-header.module.css';
56

67
interface PipelineRunLogHeaderProps {
@@ -127,7 +128,7 @@ export function PipelineExecutionLogHeader({
127128
{/* Show current step if available and different from agent name */}
128129
{step && step !== agent && (
129130
<span className="text-xs font-medium bg-emerald-500/10 text-emerald-400 px-2 py-0.5 rounded whitespace-nowrap">
130-
{step}
131+
{normalizeStepName(step)}
131132
</span>
132133
)}
133134
</div>
@@ -146,7 +147,7 @@ export function PipelineExecutionLogHeader({
146147

147148
{/* Step */}
148149
<span className="px-1.5 py-0.5 rounded bg-gray-800 text-gray-300 font-medium border border-gray-700/50 text-[0.65rem]">
149-
{step || "Step NA"}
150+
{step ? normalizeStepName(step) : "Step NA"}
150151
</span>
151152

152153
<svg className="w-2.5 h-2.5 text-gray-600 flex-shrink-0" fill="none" viewBox="0 0 24 24" stroke="currentColor">

uapi/components/base/bitcode/execution/pipeline-execution-log.tsx

Lines changed: 201 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@ import {
1111
ExclamationTriangleIcon,
1212
InfoCircledIcon,
1313
ChevronRightIcon,
14+
ListBulletIcon,
1415
} from '@radix-ui/react-icons';
1516
import FileDiffViewer from './FileDiffViewer';
1617
import type { FileDiff, FileTreeChange } from '@bitcode/streams';
@@ -362,6 +363,162 @@ export function buildRawLogCopyText(args: {
362363
].join('');
363364
}
364365

366+
// "Copy terse logs" string budgets: ordinary string fields truncate to
367+
// TERSE_STRING_LIMIT; fields whose key looks error-ish (error/message/stack)
368+
// keep TERSE_ERROR_STRING_LIMIT so failure forensics survive the distillation.
369+
const TERSE_STRING_LIMIT = 200;
370+
const TERSE_ERROR_STRING_LIMIT = 2000;
371+
const TERSE_ERROR_KEY = /error|message|stack/i;
372+
373+
/**
374+
* Recursively distill a copy payload for the "Copy terse logs" button: every
375+
* string over its budget is truncated to a preview + '… [+N chars]' marker,
376+
* while structure, ordering, counts, numbers, and short fields (the run's
377+
* phase/agent/step/failsafe hierarchy, statuses, usage, timestamps) survive
378+
* whole. Pure + exported for unit testing.
379+
*/
380+
export function distillTerseValue(value: unknown, keyHint?: string): unknown {
381+
if (typeof value === 'string') {
382+
const limit = keyHint && TERSE_ERROR_KEY.test(keyHint) ? TERSE_ERROR_STRING_LIMIT : TERSE_STRING_LIMIT;
383+
return value.length > limit ? `${value.slice(0, limit)}… [+${value.length - limit} chars]` : value;
384+
}
385+
if (Array.isArray(value)) return value.map((item) => distillTerseValue(item, keyHint));
386+
if (value && typeof value === 'object') {
387+
const out: Record<string, unknown> = {};
388+
for (const [key, entry] of Object.entries(value as Record<string, unknown>)) {
389+
out[key] = distillTerseValue(entry, key);
390+
}
391+
return out;
392+
}
393+
return value;
394+
}
395+
396+
/**
397+
* Compact one streamed run event ({id, created_at, event} or a bare payload)
398+
* into a terse row: timestamp, canonical type, store identity (namespace/key),
399+
* the full Phase→Agent→Step→Failsafe→Generation call chain + repair markers,
400+
* provider/model/usage, a bounded message preview, and (near-)complete error
401+
* bodies. Everything else — the raw stored values, executionState duplicates,
402+
* metadata snapshots — is the payload bulk and is dropped.
403+
*/
404+
export function compactTerseEvent(entry: unknown): Record<string, unknown> {
405+
const record = entry && typeof entry === 'object' ? (entry as Record<string, any>) : null;
406+
const payload = record && 'event' in record ? record.event : entry;
407+
const compact: Record<string, unknown> = {};
408+
if (record?.created_at) compact.created_at = record.created_at;
409+
if (!payload || typeof payload !== 'object') {
410+
if (payload !== undefined && payload !== null) compact.value = distillTerseValue(payload);
411+
return compact;
412+
}
413+
if (payload.type) compact.type = payload.type;
414+
// Store identity: names WHAT was stored — tiny and load-bearing for ordering
415+
// forensics (which agent/step/failsafe emitted what, in what sequence).
416+
if (payload.namespace) compact.namespace = payload.namespace;
417+
if (payload.key) compact.key = payload.key;
418+
const executionState = extractExecutionState(payload) || {};
419+
const CHAIN_FIELDS = [
420+
'pipeline',
421+
'phase',
422+
'agent',
423+
'step',
424+
'failsafe',
425+
'generation',
426+
'tool',
427+
'ptrrStepName',
428+
'promptTemplateId',
429+
'outputSchema',
430+
'returnType',
431+
] as const;
432+
for (const field of CHAIN_FIELDS) {
433+
const value = executionState[field];
434+
if (value !== undefined && value !== null && value !== '') compact[field] = value;
435+
}
436+
if (typeof executionState.stitchIteration === 'number') compact.stitchIteration = executionState.stitchIteration;
437+
if (typeof executionState.chunkIndex === 'number') compact.chunkIndex = executionState.chunkIndex;
438+
if (executionState.chunkSum === true) compact.chunkSum = true;
439+
const status = payload.status || {};
440+
// The source-safe stream projection carries its metadata under `data`
441+
// (provider/model/tool/phase/agent/step/generation, plus contentChars for
442+
// withheld bodies); `llm:usage` store events carry the usage object AS data.
443+
const data = payload.data && typeof payload.data === 'object' ? payload.data : {};
444+
for (const field of ['phase', 'agent', 'step', 'generation', 'tool'] as const) {
445+
if (compact[field] === undefined && data[field] !== undefined && data[field] !== null && data[field] !== '') {
446+
compact[field] = data[field];
447+
}
448+
}
449+
const provider = payload.provider ?? status.provider ?? data.provider ?? executionState.provider;
450+
const model = payload.model ?? status.model ?? data.model ?? executionState.model;
451+
const usage =
452+
payload.usage ??
453+
status.usage ??
454+
(payload.namespace === 'llm' && payload.key === 'usage' ? payload.data : undefined) ??
455+
executionState.usage;
456+
if (provider) compact.provider = provider;
457+
if (model) compact.model = model;
458+
if (usage !== undefined && usage !== null) compact.usage = distillTerseValue(usage);
459+
if (typeof data.contentChars === 'number') compact.contentChars = data.contentChars;
460+
if (typeof data.ok === 'boolean') compact.ok = data.ok;
461+
const message = payload.message ?? status.message ?? payload.text;
462+
// 'message' keyHint: event messages carry stall/failure text, so they get
463+
// the larger error budget the __terse note promises for message fields.
464+
if (typeof message === 'string' && message) compact.message = distillTerseValue(message, 'message');
465+
const errorBody = payload.error ?? status.error;
466+
if (errorBody !== undefined && errorBody !== null) compact.error = distillTerseValue(errorBody, 'error');
467+
return compact;
468+
}
469+
470+
/**
471+
* Build the text the "Copy terse logs" button copies: the same run payload as
472+
* "Copy raw logs", distilled to a much smaller but still debugging-useful
473+
* form. When `copyData` carries an `events` array (the /deposit shape), every
474+
* event compacts to its terse row (`compactTerseEvent`) and the
475+
* `outputDetails` duplication is omitted; other payload fields keep their
476+
* structure with long strings truncated (`distillTerseValue`) — error bodies
477+
* keep a much larger budget. Pure + exported for unit testing.
478+
*/
479+
export function buildTerseLogCopyText(args: {
480+
copyData?: unknown;
481+
output?: string;
482+
outputDetails?: Record<string, any>;
483+
error?: string | null;
484+
}): string {
485+
const { copyData, output, outputDetails, error } = args;
486+
const note =
487+
`Terse copy: run events are compacted to timestamp/type/call-chain/usage/error rows and long strings ` +
488+
`are truncated to a preview + '… [+N chars]' (error/message/stack fields keep up to ` +
489+
`${TERSE_ERROR_STRING_LIMIT} chars); ordering and counts are complete. Use 'Copy raw logs' for full bodies.`;
490+
if (
491+
copyData !== undefined &&
492+
copyData &&
493+
typeof copyData === 'object' &&
494+
!Array.isArray(copyData) &&
495+
Array.isArray((copyData as Record<string, any>).events)
496+
) {
497+
const { events, outputDetails: duplicatedDetails, ...header } = copyData as Record<string, any>;
498+
void duplicatedDetails;
499+
const wrapped = {
500+
__terse: note,
501+
...(distillTerseValue(header) as Record<string, unknown>),
502+
outputDetails: '[omitted — duplicates the events; use Copy raw logs for full bodies]',
503+
eventCount: events.length,
504+
firstEventAt: events[0]?.created_at ?? null,
505+
lastEventAt: events[events.length - 1]?.created_at ?? null,
506+
events: events.map(compactTerseEvent),
507+
};
508+
return JSON.stringify(wrapped, null, 2);
509+
}
510+
const source =
511+
copyData !== undefined
512+
? copyData
513+
: { output: output || '', outputDetails: outputDetails ?? {}, error: error ?? null };
514+
const distilled = distillTerseValue(typeof source === 'string' ? { output: source } : source);
515+
const wrapped =
516+
distilled && typeof distilled === 'object' && !Array.isArray(distilled)
517+
? { __terse: note, ...(distilled as Record<string, unknown>) }
518+
: { __terse: note, data: distilled };
519+
return JSON.stringify(wrapped, null, 2);
520+
}
521+
365522
// Matches the default BITCODE_LLM_CALL_TIMEOUT_MS (AgentLLMsRegistry /
366523
// PipelineLLMRegistry) — past this many seconds with no new row, an in-flight
367524
// LLM call should already have timed out server-side, so continued silence is
@@ -471,6 +628,19 @@ export const PipelineExecutionLog = forwardRef<HTMLDivElement, PipelineRunLogPro
471628
}
472629
};
473630

631+
// "Copy terse logs": the same run payload distilled — long strings truncated,
632+
// hierarchy/ordering/errors kept — for a much smaller but still useful copy.
633+
const [copiedTerse, setCopiedTerse] = useState(false);
634+
const handleCopyTerse = async () => {
635+
const ok = await copyTextToClipboard(
636+
buildTerseLogCopyText({ copyData, output, outputDetails, error }),
637+
);
638+
if (ok) {
639+
setCopiedTerse(true);
640+
setTimeout(() => setCopiedTerse(false), 1500);
641+
}
642+
};
643+
474644
useLayoutEffect(() => {
475645
if (typeof window === 'undefined' || typeof ResizeObserver === 'undefined') return;
476646
if (!containerRef.current) return;
@@ -828,19 +998,34 @@ export const PipelineExecutionLog = forwardRef<HTMLDivElement, PipelineRunLogPro
828998

829999
return (
8301000
<div className="relative w-full">
831-
<button
832-
type="button"
833-
onClick={handleCopyRaw}
834-
title="Copy raw logs"
835-
aria-label="Copy raw logs"
836-
className="absolute top-2 right-2 z-30 flex h-7 w-7 items-center justify-center rounded-md border border-white/10 bg-black/40 text-neutral-300 backdrop-blur-sm transition hover:border-emerald-300/40 hover:text-emerald-200 focus:outline-none"
837-
>
838-
{copiedRaw ? (
839-
<CheckIcon className="h-4 w-4 text-emerald-300" />
840-
) : (
841-
<ClipboardCopyIcon className="h-4 w-4" />
842-
)}
843-
</button>
1001+
<div className="absolute top-2 right-2 z-30 flex items-center gap-1">
1002+
<button
1003+
type="button"
1004+
onClick={handleCopyTerse}
1005+
title="Copy terse logs"
1006+
aria-label="Copy terse logs"
1007+
className="flex h-7 w-7 items-center justify-center rounded-md border border-white/10 bg-black/40 text-neutral-300 backdrop-blur-sm transition hover:border-emerald-300/40 hover:text-emerald-200 focus:outline-none"
1008+
>
1009+
{copiedTerse ? (
1010+
<CheckIcon className="h-4 w-4 text-emerald-300" />
1011+
) : (
1012+
<ListBulletIcon className="h-4 w-4" />
1013+
)}
1014+
</button>
1015+
<button
1016+
type="button"
1017+
onClick={handleCopyRaw}
1018+
title="Copy raw logs"
1019+
aria-label="Copy raw logs"
1020+
className="flex h-7 w-7 items-center justify-center rounded-md border border-white/10 bg-black/40 text-neutral-300 backdrop-blur-sm transition hover:border-emerald-300/40 hover:text-emerald-200 focus:outline-none"
1021+
>
1022+
{copiedRaw ? (
1023+
<CheckIcon className="h-4 w-4 text-emerald-300" />
1024+
) : (
1025+
<ClipboardCopyIcon className="h-4 w-4" />
1026+
)}
1027+
</button>
1028+
</div>
8441029
<div
8451030
ref={(node) => {
8461031
containerRef.current = node;
@@ -1448,10 +1633,11 @@ function renderLogLine(
14481633
try {
14491634
const stores = logLine.details?.status?.metadata?.stores || logLine.details?.metadata?.stores;
14501635
const stepLower = String(logLine.step || '').toLowerCase();
1636+
// 'retry' must be tested before 'try' ('retry'.includes('try')).
14511637
const stepName = stepLower.includes('plan') ? 'plan'
1638+
: stepLower.includes('retry') || stepLower.includes('intensify') ? 'retry'
14521639
: stepLower.includes('try') || stepLower.includes('generate') ? 'try'
14531640
: stepLower.includes('refine') ? 'refine'
1454-
: stepLower.includes('retry') || stepLower.includes('intensify') ? 'retry'
14551641
: undefined;
14561642
if (!stores || !logLine.phase || !logLine.agent || !stepName) return null;
14571643
const vm = buildStepViewModel({ phase: logLine.phase, agent: logLine.agent, step: stepName as any }, stores);
@@ -1465,7 +1651,7 @@ function renderLogLine(
14651651
<div>
14661652
<span className="text-gray-500 mr-1">Failsafes:</span>
14671653
{vm.failsafes.map(f => (
1468-
<span key={f.failsafe} className="inline-block mr-2 text-emerald-300">{f.failsafe}</span>
1654+
<span key={f.failsafe} className="inline-block mr-2 text-emerald-300">{formatMeta(f.failsafe)}</span>
14691655
))}
14701656
</div>
14711657
{vm.tools.used.length > 0 && (

0 commit comments

Comments
 (0)