Skip to content

Commit a73f5e4

Browse files
V48: QA Deposit Clone-Vcs Agent End
Marker at refine/CS/structured_output (49 calls). Fix useTools hoist, clone tool registration + git-https, PCC host ranking, thin VCS, and workspacePath merge after tools. Ledger 1.D9 Accepted.
1 parent 03830bd commit a73f5e4

15 files changed

Lines changed: 612 additions & 99 deletions

.qa/BITCODE_V48_CANONICAL_PROMOTION_ACCEPTANCE.md

Lines changed: 93 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -29,8 +29,8 @@
2929

3030
| § | Topic | Status |
3131
| --- | --- | --- |
32-
| 1 | Every-call / every-pipeline LLM debug | **Partial** (read 1.1; deposit 1.D1–1.D8; **Try PCC reason**) |
33-
| 2 | SDIVF deposit pipeline production-like accept | **Partial** (clone-vcs Plan closed; Try PCC started) |
32+
| 1 | Every-call / every-pipeline LLM debug | **Partial** (read 1.1; deposit 1.D1–1.D9; **clone-vcs agent end**) |
33+
| 2 | SDIVF deposit pipeline production-like accept | **Partial** (clone-vcs Plan→Try→Retry→Refine closed at agent end) |
3434
| 3 | SDIVF read pipeline production-like accept | Open (partial offline via §1.1) |
3535
| 4 | Settle Simple pipeline production-like accept | Open |
3636
| 5 | Discovery law (wave-1 parallel → product search keys) | Open |
@@ -75,10 +75,11 @@ pnpm --filter @bitcode/pipeline-hosts run qa:read:debug-first-llm
7575
| `BITCODE_DEBUG_FORCE_CLONE_PTRR` | `1` | Force real clone PTRR agent |
7676
| `BITCODE_DEBUG_STOP_AFTER_FIRST_REASON` | `1` | Hard-stop **flag** name (historical); generation pin is separate |
7777
| `BITCODE_DEBUG_STOP_PHASE` | `setup` | |
78-
| `BITCODE_DEBUG_STOP_STEP` | **`try`** | progressive Try (after 1.D7 Plan close) |
79-
| `BITCODE_DEBUG_STOP_FAILSAFE` | **`prepare_concise_context`** | Try PCC selection |
80-
| `BITCODE_DEBUG_STOP_GENERATION` | **`reason`** | Try PCC reason (1.D8) |
78+
| `BITCODE_DEBUG_STOP_STEP` | **`refine`** | end of clone-vcs agent (after 1.D9) |
79+
| `BITCODE_DEBUG_STOP_FAILSAFE` | **`chunk_then_sum`** | Refine CS task SO |
80+
| `BITCODE_DEBUG_STOP_GENERATION` | **`structured_output`** | agent terminal task gen |
8181
| `BITCODE_DEBUG_STOP_AGENT_FILTER` | `clone-vcs` | |
82+
| `BITCODE_PIPELINE_HOST_MAX_RUNTIME_MS` | `900000` | full Plan+Try+Retry+Refine budget |
8283
| `BITCODE_LLM_PROVIDER` / `BITCODE_LLM_MODEL` | anthropic / `claude-haiku-4-5` | |
8384

8485
**Artifact root:** `.tmp/llm-call-debug/<runId>/`
@@ -1322,26 +1323,104 @@ Explicit: “No tools are invoked in this Reason generation.”
13221323

13231324
| | |
13241325
| --- | --- |
1325-
| **this stop** | **Accepted** |
1326-
| **next marker** | Try · prepare_concise_context · **judge** |
1327-
| **not yet** | Try PCC SO → CS → useTools → clone tool; remaining Setup |
1326+
| **this stop** | **Accepted** (historical) |
1327+
| **next marker** | advanced → full agent end (see **1.D9**) |
1328+
| **not yet** | (closed by 1.D9) remaining Setup agents |
13281329

13291330
#### artifacts
13301331

13311332
| Kind | Path |
13321333
| --- | --- |
13331334
| Request | `.tmp/llm-call-debug/…/0013-request-…-try-prepare_concise_context-reason.json` |
1334-
| Response | `.tmp/llm-call-debug/…/0014-response-…-try-prepare_concise_context-reason.json` |
1335+
| Response | `.tmp/llm-call-debug/…/0014-response-…-try-prepare_concise-context-reason.json` |
13351336
| Abort | `.tmp/llm-call-debug/…/0015-abort-…-try-…-reason.json` |
13361337
| Re-run | `pnpm --filter @bitcode/pipeline-hosts run qa:deposit:debug-first-llm` |
13371338

13381339
---
13391340

1341+
### 1.D9 Deposit · Setup · clone-vcs · **Agent end** (Plan→Try→Retry→Refine CS SO)
1342+
1343+
- **status:** **Accepted (clone-vcs agent complete under progressive marker)**
1344+
- **date:** 2026-07-16
1345+
- **commit_tag:** `QA Pipeline Deposit Phase Setup Agent Clone-Vcs Agent End Plan Try Retry Refine`
1346+
- **pipeline:** `ExecutionPipelineSDIVFSynthesizeDepositAssetPacks`
1347+
- **phase:** setup
1348+
- **agent:** `asset-pack-clone-vcs-repository-agent`
1349+
- **step / failsafe / thinking (stop):** **refine** / **chunk_then_sum** / **structured_output**
1350+
- **execution_path (abort):**
1351+
`…/agent:asset-pack-clone-vcs-repository-agent → refine → failsafe:chunk_then_sum → gen-0 → seq-2 → thinkings:structured_output`
1352+
- **provider / model:** anthropic / `claude-haiku-4-5`
1353+
- **harness:** `ok: true`, `debugStop: true`, **`callCount: 49`**, stopStep=`refine`, stopFailsafe=`chunk_then_sum`, stopGeneration=`structured_output`
1354+
- **abort:** `hard-stop after refine/chunk_then_sum/structured_output agent=asset-pack-clone-vcs-repository-agent`
1355+
1356+
#### coverage (single agent-end run)
1357+
1358+
| PTRR step | PCC (R/J/SO) | CS (R/J/SO) | Tools postprocess |
1359+
| --- | --- | --- | --- |
1360+
| **Plan** || ✓ PlanStepOutput (approach/steps) | none (correct) |
1361+
| **Try** || ✓ agent schema + **useTools** | **yes**`asset-pack-clone-vcs-repository-tool` |
1362+
| **Retry** | ✓ (includes try tools:used keys) | ✓ real `workspacePath` under `/var/folders/…/bitcode-clone-…` | yes when selected |
1363+
| **Refine** || ✓ agent return (stop here) | none (correct; no postprocess) |
1364+
1365+
Stitch: non-triggering (schema-valid CS SO; stitchCount=0).
1366+
1367+
#### critique → fix (this agent session)
1368+
1369+
| Defect | Fix |
1370+
| --- | --- |
1371+
| VCS system “DevOps soup” (Jenkins/Gerrit) | Thin Host-bound VCS prose |
1372+
| PCC ranking preferred null deposit shells | Prefer `#host:sourceRevision` / workspace; Plan PCC SO now selects host keys first |
1373+
| Try/Retry `useTools` never executed | (1) hoist `reasoning.useTools` → tools postprocess; (2) add optional `useTools` on clone agent schema; (3) **tool not registered** — missing `__docCodePrompt` aborted whole tool loop; per-tool fail-soft + attach prompt |
1374+
| Clone tool `undefined.length` on agent-shaped input | Rewrite AssetPack clone tool to accept `{provider,owner,name,ref}` and git-https clone |
1375+
| Try SO stuck pre-tool | Tools postprocess folds successful `workspacePath` into step `output` |
1376+
| Settlement/delivery on synth | Already purged (no `settleDelivery` / `settlementBoundary` on wire) |
1377+
1378+
#### excellence (agent end)
1379+
1380+
| Axis | Result |
1381+
| --- | --- |
1382+
| **hierarchy** | **Pass** — Pipeline→Phase→Agent→Step→Failsafe→Thinking; Execution once |
1383+
| **dual law** | **Pass** — PCC keys-only user; CS prepared `selectedKeys`+`selectedContext` |
1384+
| **useTools law** | **Pass** — Plan/PCC omit; Try reason+SO select catalog tool; Refine no postprocess |
1385+
| **tools execute** | **Pass** — postprocess runs; real temp workspace from clone tool |
1386+
| **settle ban** | **Pass** — no settleDelivery on synth wire |
1387+
| **prompt hygiene** | **Pass** — thin VCS; no Jenkins soup |
1388+
1389+
#### residual (non-blocking for agent-end accept)
1390+
1391+
- Refine CS may still invent wrong tool name (`cloneRepositoryTool`) or pending status in SO text (no tools postprocess on Refine — final return quality, not execution).
1392+
- Thin VCS string intermittent on some system walks (agent still has VCS Operations identity).
1393+
- Many non-clone tools still lack `__docCodePrompt` (logged skip at preprocess; catalog partially populated).
1394+
- Next progressive marker: next Setup agent or Discovery (not further clone-vcs gens).
1395+
1396+
#### stability_confidence
1397+
1398+
- **0.90** that clone-vcs agent end (Plan→Try→Retry→Refine CS SO, 49 calls, tools postprocess) is successful for V48 progressive deposit QA.
1399+
1400+
#### decision
1401+
1402+
| | |
1403+
| --- | --- |
1404+
| **this stop** | **Accepted — clone-vcs agent complete** |
1405+
| **next marker** | next Setup agent **or** Discovery (leave refine/CS/SO until next agent starts) |
1406+
| **not yet** | initialize-lsp / mcps / comprehend-obfuscations / danger-wall; Discovery |
1407+
1408+
#### artifacts
1409+
1410+
| Kind | Path |
1411+
| --- | --- |
1412+
| Work root | `.tmp/local-deposit-debug/` |
1413+
| Wire report | `.tmp/llm-call-debug/pipeline-synthesize_deposit_asset_packs/VERBATIM_WIRE_REPORT.md` |
1414+
| Ledger | `…/ledger.jsonl` (49 entries) |
1415+
| Abort | `…/0049-abort-…-refine-chunk_then_sum-structured_output.json` |
1416+
| Re-run | `pnpm --filter @bitcode/pipeline-hosts run qa:deposit:debug-first-llm` |
1417+
1418+
---
1419+
13401420
### 1.2+ Read next / Deposit next
13411421

1342-
- **1.D9** Deposit · Setup · clone-vcs · **Try** · PCC · **judge**
1422+
- **1.D10** next Setup agent (or Discovery) — progressive marker
13431423
- **1.2** Read · Setup · clone-vcs · Plan · PCC · **judge**
1344-
- **1.D10+** Try PCC SO → CS → tools/clone; remaining Setup; Discovery · …
13451424

13461425
---
13471426

@@ -1351,8 +1430,8 @@ Explicit: “No tools are invoked in this Reason generation.”
13511430

13521431
- **status:** **Partial**
13531432
- **criterion:** full Setup→…→Finish deposit run under LocalHost / production-like accept with real inference; until then, §1 deposit call-by-call rows are the progressive proof.
1354-
- **proof (current):** §1.D1–1.D7 Plan complete; §1.D8 Try PCC reason Accepted.
1355-
`pnpm --filter @bitcode/pipeline-hosts run qa:deposit:debug-first-llm``debugStop: true`, stopStep=`try`, stopGeneration=`reason`, callCount=15.
1433+
- **proof (current):** §1.D1–1.D9 **clone-vcs agent complete** (Plan→Try→Retry→Refine, tools postprocess, callCount=49).
1434+
`pnpm --filter @bitcode/pipeline-hosts run qa:deposit:debug-first-llm``debugStop: true`, stopStep=`refine`, stopFailsafe=`chunk_then_sum`, stopGeneration=`structured_output`.
13561435

13571436
### §3 SDIVF read pipeline production-like accept
13581437

@@ -1418,3 +1497,4 @@ Explicit: “No tools are invoked in this Reason generation.”
14181497
| --- | --- |
14191498
| 2026-07-16 | Document created. §1 method + entry **1.1** (read clone Plan PCC reason) filled from live `pnpm run debug:read:first-llm` after commits retiring substeps, PTRR order, and PCC prompt work. |
14201499
| 2026-07-16 | §7 retitled: Host selection (API/dispatch) **vs** SDIVF Setup phase — naming clarification only; no host-selection bug found in read synthesize path from this audit. |
1500+
| 2026-07-16 | **1.D9** clone-vcs agent end Accepted: marker refine/CS/SO; useTools hoist + clone tool registration/git-https; tools postprocess merges workspacePath. |

packages/agent-generics/src/__tests__/step-postprocess-tools.test.ts

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,4 +24,32 @@ describe('Tools execute as Step postprocess once', () => {
2424

2525
spy.mockRestore();
2626
});
27+
28+
it('runs tools when useTools is only on reasoning (SO omitted)', async () => {
29+
const generationFactories = require('../generations/llm-bound-factories');
30+
const spy = jest.spyOn(generationFactories, 'factoryToolsExecution').mockImplementation(() => {
31+
return async (input: any) => ({ ...input, usedTools: [{ tool: 'from-reason' }] });
32+
});
33+
34+
const step = factoryTryStep<any, any>({ parse: (x: any) => x } as any);
35+
const exec = new Execution('agent-root');
36+
37+
const out = await step(
38+
{
39+
output: { success: false },
40+
reasoning: {
41+
analysis: 'need clone',
42+
reasoningItems: [],
43+
conclusion: 'run tool',
44+
confidence: 0.9,
45+
useTools: [{ name: 'asset-pack-clone-vcs-repository-tool', input: { owner: 'o', name: 'n' } }],
46+
},
47+
},
48+
exec as any,
49+
);
50+
expect(spy).toHaveBeenCalled();
51+
expect(out.usedTools).toEqual([{ tool: 'from-reason' }]);
52+
53+
spy.mockRestore();
54+
});
2755
});
Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
// @ts-nocheck
2+
/**
3+
* Cross-registry tool lookup: pipeline ExecutionPipelineToolRegistry
4+
* must be visible from StepExecution AgentToolsRegistry after restrictTo.
5+
*/
6+
import { Execution } from '@bitcode/execution-generics';
7+
import { Tool } from '@bitcode/tools-generics';
8+
import { AgentExecution, StepExecution, applyStepToolSurface } from '../execution';
9+
import { factoryToolsExecution } from '../generations/llm-bound-factories';
10+
11+
class CloneTool extends Tool {
12+
name = 'asset-pack-clone-vcs-repository-tool';
13+
use = jest.fn(async (input: any) => ({ ok: true, input }));
14+
}
15+
16+
/** Minimal stand-in for ExecutionPipelineToolRegistry (same surface). */
17+
class FakePipelineTools {
18+
private map = new Map<string, any>();
19+
constructor(public execution: any) {}
20+
registerTool(key: string, tool: any) { this.map.set(key, tool); }
21+
get(key: string) { return this.map.get(key); }
22+
getTool(key: string) { return this.map.get(key); }
23+
getPaths() { return [...this.map.keys()]; }
24+
getUsableTools() {
25+
const o: any = {};
26+
for (const k of this.map.keys()) o[k] = this.map.get(k);
27+
return o;
28+
}
29+
}
30+
31+
describe('cross-registry tool hierarchy', () => {
32+
it('step getTool finds pipeline-registered tool under restrictTo', () => {
33+
const pipeline = new Execution('pipeline') as any;
34+
pipeline.tools = new FakePipelineTools(pipeline);
35+
const clone = new CloneTool();
36+
pipeline.tools.registerTool('asset-pack-clone-vcs-repository-tool', clone);
37+
38+
const agent = new AgentExecution('agent:clone', pipeline);
39+
const tryStep = new StepExecution('try', agent);
40+
applyStepToolSurface(tryStep, ['asset-pack-clone-vcs-repository-tool']);
41+
42+
expect(tryStep.tools.getTool('asset-pack-clone-vcs-repository-tool')).toBeTruthy();
43+
expect(Object.keys(tryStep.tools.getUsableTools())).toEqual([
44+
'asset-pack-clone-vcs-repository-tool',
45+
]);
46+
});
47+
48+
it('tools postprocess on sequential child still finds tool', async () => {
49+
const pipeline = new Execution('pipeline') as any;
50+
pipeline.tools = new FakePipelineTools(pipeline);
51+
const clone = new CloneTool();
52+
pipeline.tools.registerTool('asset-pack-clone-vcs-repository-tool', clone);
53+
54+
const agent = new AgentExecution('agent:clone', pipeline);
55+
const tryStep = new StepExecution('try', agent);
56+
applyStepToolSurface(tryStep, ['asset-pack-clone-vcs-repository-tool']);
57+
58+
// Mimic sequential(core, tools) → tools runs on child seq-1
59+
const seq1 = tryStep.child('seq-1');
60+
const toolsExec = factoryToolsExecution();
61+
const out = await toolsExec(
62+
{
63+
output: {
64+
useTools: [
65+
{
66+
name: 'asset-pack-clone-vcs-repository-tool',
67+
input: { owner: 'a', name: 'b' },
68+
},
69+
],
70+
},
71+
},
72+
seq1 as any,
73+
);
74+
expect(out.usedTools?.[0]?.error).toBeUndefined();
75+
expect(out.usedTools?.[0]?.tool).toBe('asset-pack-clone-vcs-repository-tool');
76+
expect(clone.use).toHaveBeenCalled();
77+
});
78+
});

0 commit comments

Comments
 (0)