Skip to content

Commit d34c25b

Browse files
V48 (spec-impl): QA Codebase Agent Closed
Accept Try/Retry/Refine with multi-tool useTools. Next regurgitation.
1 parent 577c800 commit d34c25b

5 files changed

Lines changed: 142 additions & 19 deletions

File tree

.qa/BITCODE_V48_CANONICAL_PROMOTION_ACCEPTANCE.md

Lines changed: 111 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -29,8 +29,8 @@
2929

3030
| § | Topic | Status |
3131
| --- | --- | --- |
32-
| 1 | Every-call / every-pipeline LLM debug | **Partial** (Discovery codebase **Plan step closed** 1.D14–1.D22; next Try 1.D23) |
33-
| 2 | SDIVF deposit pipeline production-like accept | **Partial** (Setup closed; Discovery Plan step closed) |
32+
| 1 | Every-call / every-pipeline LLM debug | **Partial** (Discovery codebase **agent closed** 1.D14–1.D-Agent; next inherent-regurgitation) |
33+
| 2 | SDIVF deposit pipeline production-like accept | **Partial** (Setup closed; first Discovery agent closed) |
3434
| 3 | SDIVF read pipeline production-like accept | Open (partial offline via §1.1) |
3535
| 4 | Settle Simple pipeline production-like accept | Open |
3636
| 5 | Discovery law (wave-1 parallel → product search keys) | Open |
@@ -75,14 +75,14 @@ pnpm --filter @bitcode/pipeline-hosts run qa:read:debug-first-llm
7575
| `BITCODE_DEBUG_FORCE_CLONE_PTRR` | `1` | Force real clone PTRR agent |
7676
| `BITCODE_DEBUG_STOP_AFTER_FIRST_REASON` | `1` | Hard-stop **flag** name (historical); generation pin is separate |
7777
| `BITCODE_DEBUG_STOP_PHASE` | **`discovery`** | Setup closed |
78-
| `BITCODE_DEBUG_STOP_STEP` | **`try`** | Plan closed; next Try PCC reason |
79-
| `BITCODE_DEBUG_STOP_FAILSAFE` | **`prepare_concise_context`** | Try PCC |
80-
| `BITCODE_DEBUG_STOP_GENERATION` | **`reason`** | 1.D23 Try PCC reason |
78+
| `BITCODE_DEBUG_STOP_STEP` | **`plan`** | next agent Plan PCC reason |
79+
| `BITCODE_DEBUG_STOP_FAILSAFE` | **`prepare_concise_context`** | PCC |
80+
| `BITCODE_DEBUG_STOP_GENERATION` | **`reason`** | first Thinkings |
8181
| `BITCODE_DEBUG_FORCE_CLONE_PTRR` | **`0`** | clone PTRR closed at 1.D9; host-adopt |
8282
| `BITCODE_DEBUG_SETUP_SERIAL` | **`1`** | serial Setup wave-1 |
8383
| `BITCODE_DEBUG_FAST_SETUP` | **`1`** | Discovery QA: skip re-PTRR obfuscations/LSP |
8484
| `BITCODE_DEBUG_DISCOVERY_SERIAL` | **`1`** | serial Discovery wave-1 (no parallel race) |
85-
| `BITCODE_DEBUG_STOP_AGENT_FILTER` | **`DepositCodebaseComprehension`** | first Discovery agent |
85+
| `BITCODE_DEBUG_STOP_AGENT_FILTER` | **`DepositInherentRegurgitation`** | second wave-1 agent |
8686
| `BITCODE_PIPELINE_HOST_MAX_RUNTIME_MS` | `900000` | full Plan+Try+Retry+Refine budget |
8787
| `BITCODE_LLM_PROVIDER` / `BITCODE_LLM_MODEL` | anthropic / `claude-haiku-4-5` | |
8888

@@ -1559,7 +1559,7 @@ Stitch: non-triggering (schema-valid CS SO; stitchCount=0).
15591559
- `BITCODE_DEBUG_SETUP_SERIAL=1` for progressive stops
15601560
- `BITCODE_LLM_CALL_TIMEOUT_MS=0` when Judge/CS can exceed 180s
15611561
- **layout fixes during Setup QA:** host import split (all-3 / synth / product); zod on read package; serial Setup debug flag; discovery registration import path; codebase-analysis-helpers package export
1562-
- **next progressive marker:** Discovery · `DepositCodebaseComprehensionAgent` **Try** PCC reason (1.D23)
1562+
- **next progressive marker:** Discovery · `DepositInherentRegurgitationAgent` Plan PCC reason
15631563

15641564
### 1.D14 Deposit · Discovery · comprehend-codebase · Plan · PCC · reason
15651565

@@ -1695,11 +1695,110 @@ Stitch: non-triggering (schema-valid CS SO; stitchCount=0).
16951695
| 6 | 1.D19 | CS SO | Accepted |
16961696
| 7–9 | 1.D20–22 | Stitch | Zero-LLM fence Accepted |
16971697

1698+
### 1.D23 Deposit · Discovery · comprehend-codebase · **Try** · PCC · reason
1699+
1700+
- **status:** **Accepted**
1701+
- **date:** 2026-07-17
1702+
- **usage:** ~10.1k in / ~944 out
1703+
- **role:** Pass — Try PCC key selection; prioritizes host coords, source-safety, **sourceCheckoutCatalog**, **#discovery:usableToolNames**, workspacePath
1704+
- **useTools:** omitted (correct for PCC)
1705+
- **stability_confidence:** **0.92**
1706+
- **decision:** advance to Try PCC judge
1707+
- **commit_tag:** QA Deposit Discovery Codebase Try Pcc Reason Call-Site
1708+
1709+
### 1.D24 Deposit · Discovery · comprehend-codebase · Try · PCC · judge
1710+
1711+
- **status:** **Accepted**
1712+
- **date:** 2026-07-17
1713+
- **usage:** ~10.7k in / ~589 out
1714+
- **completion:** Judgment quality ~0.78; critiques on path form / host runId — call-site valid
1715+
- **stability_confidence:** **0.92**
1716+
- **commit_tag:** QA Deposit Discovery Codebase Try Pcc Judge Call-Site
1717+
1718+
### 1.D25 Deposit · Discovery · comprehend-codebase · Try · PCC · structured_output
1719+
1720+
- **status:** **Accepted** (Try PCC selection complete)
1721+
- **date:** 2026-07-17
1722+
- **usage:** 11363 in / 102 out
1723+
- **selectedKeys (8):** `#host:manifestRoot`, `#host:sourceRevision`, deposit obfuscations + path gates + **sourceCheckoutCatalog**, `#repository:workspacePath`, **`#discovery:usableToolNames`**
1724+
- **no useTools** (PCC law)
1725+
- **stability_confidence:** **0.93**
1726+
- **commit_tag:** QA Deposit Discovery Codebase Try Pcc Structured-Output Call-Site
1727+
1728+
### 1.D26 Deposit · Discovery · comprehend-codebase · Try · CS · reason
1729+
1730+
- **status:** **Accepted** — multi-tool Try strategy **proven**
1731+
- **date:** 2026-07-17
1732+
- **usage:** ~11.2k in / ~2990 out (large useTools payload)
1733+
- **useTools:** **18** calls — `host-workspace-read-file`×5, `host-workspace-run-command`×4, `host-workspace-list-dir`×2, `lsp-workspace-symbols`×2, `lsp-document-symbols`×2, `lsp-hover`/`definition`/`references`
1734+
- **diversity:** same tools with different paths/queries (index.js, test.js, index.d.ts, package.json, .github/, git log/show, rg, multiple LSP positions)
1735+
- **confidence:** 0.95
1736+
- **stability_confidence:** **0.94**
1737+
- **residual fixed after:** host tools accept `workspacePath` alias for `workspaceRoot` (models emit Path)
1738+
- **commit_tag:** QA Deposit Discovery Codebase Try Cs Reason Call-Site
1739+
1740+
### 1.D27 Deposit · Discovery · comprehend-codebase · Try · CS · judge
1741+
1742+
- **status:** **Accepted**
1743+
- **date:** 2026-07-17
1744+
- **usage:** ~12.3k in / ~625 out
1745+
- **quality:** ~0.88 — valid task judge on multi-tool plan
1746+
- **stability_confidence:** **0.92**
1747+
- **commit_tag:** QA Deposit Discovery Codebase Try Cs Judge Call-Site
1748+
1749+
### 1.D28 Deposit · Discovery · comprehend-codebase · Try · CS · structured_output
1750+
1751+
- **status:** **Accepted** (Try task SO + tools postprocess)
1752+
- **date:** 2026-07-17
1753+
- **usage:** ~15.4k in / ~3560 out (comprehension + useTools re-emit)
1754+
- **completion:** full `comprehension` map for is-plain-obj + **useTools 18** (hoist/postprocess)
1755+
- **role:** Pass — Try SO may include useTools; tools postprocess runs after failsafes
1756+
- **stability_confidence:** **0.93**
1757+
- **commit_tag:** QA Deposit Discovery Codebase Try Cs Structured-Output Call-Site
1758+
1759+
### 1.D29–1.D31 Deposit · Discovery · comprehend-codebase · Try · Stitch
1760+
1761+
- **status:** **Accepted (zero-LLM fence)**
1762+
- **evidence:** no stitch wire files; pipeline advanced to Retry after Try CS
1763+
- **commit_tag:** QA Deposit Discovery Codebase Try Stitch Fence Call-Site
1764+
1765+
### 1.D32–1.D37 Deposit · Discovery · comprehend-codebase · **Retry** · PCC+CS (Stitch fence)
1766+
1767+
- **status:** **Accepted** (full Retry Thinkings observed end-to-end)
1768+
- **date:** 2026-07-17
1769+
- **Retry CS reason useTools:** **12** additional (read-file×4, list-dir×2, run-command×3, LSP symbols)
1770+
- **Retry CS SO:** comprehension refreshed; residual toolsUsed field sometimes labels seed context (non-blocking)
1771+
- **Stitch:** zero-LLM
1772+
- **stability_confidence:** **0.90**
1773+
- **commit_tag:** QA Deposit Discovery Codebase Retry Step Call-Site
1774+
1775+
### 1.D38–1.D43 Deposit · Discovery · comprehend-codebase · **Refine** · PCC+CS (Stitch fence)
1776+
1777+
- **status:** **Accepted**
1778+
- **date:** 2026-07-17
1779+
- **useTools:** omitted on Refine (correct — Refine has no tools postprocess)
1780+
- **Refine CS SO:** polished source-safe knowledge map (is-plain-obj type-guard utility)
1781+
- **Stitch:** zero-LLM
1782+
- **stability_confidence:** **0.92**
1783+
- **commit_tag:** QA Deposit Discovery Codebase Refine Step Call-Site
1784+
1785+
### 1.D-Agent · **DepositCodebaseComprehensionAgent closed**
1786+
1787+
- **status:** **Closed** (2026-07-17)
1788+
- **roster:** Plan 1.D14–22 → Try 1.D23–31 → Retry 1.D32–37 → Refine 1.D38–43
1789+
- **proof highlights:**
1790+
- Multi-tool Try with **18** useTools (LSP + host read/list/run) and diversified parameters
1791+
- Retry added **12** more tool selections
1792+
- Refine polished without tools
1793+
- Host tools registry + `workspacePath` alias; full LSP suite registered for agent
1794+
- **next progressive marker:** Discovery wave-1 sibling **`DepositInherentRegurgitationAgent`** Plan PCC reason
1795+
(then wave-2 **search-depository-for-deposit-relevants**)
1796+
16981797
### 1.2+ Read next / Deposit next
16991798

1700-
- **1.D23** Discovery · comprehend-codebase · **Try** · PCC · reason
1799+
- **1.D44** Discovery · inherent-regurgitation · Plan · PCC · reason
17011800
- **1.2** Read · Setup · clone-vcs · Plan · PCC · **judge**
1702-
- **later** Discovery wave-2 · search-depository (deposit relevants / read Need-fits) + low-level tool live proof
1801+
- **later** Discovery wave-2 · search-depository (deposit relevants / read Need-fits) live tool proof
17031802

17041803
---
17051804

@@ -1709,8 +1808,8 @@ Stitch: non-triggering (schema-valid CS SO; stitchCount=0).
17091808

17101809
- **status:** **Partial**
17111810
- **criterion:** full Setup→…→Finish deposit run under LocalHost / production-like accept with real inference; until then, §1 deposit call-by-call rows are the progressive proof.
1712-
- **proof (current):** Setup closed; Discovery codebase **Plan step closed** (1.D14–1.D22).
1713-
Harness: `BITCODE_DEBUG_FAST_SETUP=1` + serial Setup/Discovery.
1811+
- **proof (current):** Setup closed; Discovery **codebase comprehension agent closed** (1.D14–1.D-Agent).
1812+
Multi-tool Try proven (18 useTools). Harness: fast Setup + serial Discovery.
17141813

17151814
### §3 SDIVF read pipeline production-like accept
17161815

@@ -1781,3 +1880,4 @@ Stitch: non-triggering (schema-valid CS SO; stitchCount=0).
17811880
| 2026-07-17 | **1.D-L / 1.D-M / 1.D-W** LSP agent end (residual uninitialized), MCP + danger-wall Accepted; **Setup phase closed**. Marker → Discovery 1.D14. |
17821881
| 2026-07-17 | **1.D14** Discovery codebase Plan PCC reason Accepted. Fast Setup + Discovery serial harness; Need-first depository search query plan. Marker → 1.D15 judge. |
17831882
| 2026-07-17 | **1.D15–1.D22** Plan step closed (PCC R/J/SO, CS R/J/SO, Stitch zero-LLM fence). Marker → Try PCC reason 1.D23. |
1883+
| 2026-07-17 | **1.D23–1.D-Agent** Codebase agent closed (Try 18 tools, Retry 12, Refine). Marker → inherent-regurgitation. |

packages/asset-packs-pipelines/syntheses/deposit/src/agents/discovery/deposit-codebase-comprehension-agent.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -130,7 +130,7 @@ const TRY = part(
130130
`(4) LSP tools (${LSP_TOOL_NAMES.workspaceSymbols}, ${LSP_TOOL_NAMES.documentSymbols},`,
131131
`${LSP_TOOL_NAMES.definition}, ${LSP_TOOL_NAMES.references}, ${LSP_TOOL_NAMES.hover})`,
132132
'with DIFFERENT path/query/line/character inputs — not a single token call.',
133-
'Always pass workspaceRoot/workspacePath from selected context into host tools.',
133+
'Always pass workspaceRoot (or workspacePath alias) from selected context into host tools.',
134134
'After tools postprocess, produce comprehension grounded in tool results + seed measurements.',
135135
'Source-safe: never dump secrets; paths only from catalog or tool results.',
136136
].join(' '),

packages/asset-packs-pipelines/syntheses/domain/src/__tests__/discovery-host-workspace-tools.test.ts

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,13 @@ describe('discovery host workspace tools', () => {
3838
expect(ok.ok).toBe(true);
3939
expect(ok.content).toContain('export const x');
4040

41+
// Models often pass workspacePath (same as Host checkout path).
42+
const alias = await runHostWorkspaceReadFile({
43+
workspacePath: root,
44+
path: 'hello.js',
45+
} as any);
46+
expect(alias.ok).toBe(true);
47+
4148
const escape = await runHostWorkspaceReadFile({
4249
workspaceRoot: root,
4350
path: '../outside',

packages/asset-packs-pipelines/syntheses/domain/src/tools/discovery-host-workspace-tools.ts

Lines changed: 19 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -30,8 +30,20 @@ function resolveUnderRoot(workspaceRoot: string, relPath: string): string | null
3030
return target;
3131
}
3232

33+
function resolveWorkspaceRoot(input: {
34+
workspaceRoot?: string;
35+
workspacePath?: string;
36+
root?: string;
37+
}): string {
38+
return String(
39+
input?.workspaceRoot || input?.workspacePath || input?.root || '',
40+
).trim();
41+
}
42+
3343
async function readFileBounded(input: {
3444
workspaceRoot?: string;
45+
workspacePath?: string;
46+
root?: string;
3547
path?: string;
3648
maxBytes?: number;
3749
}): Promise<{
@@ -43,7 +55,7 @@ async function readFileBounded(input: {
4355
error?: string;
4456
}> {
4557
const rel = String(input?.path || '').trim();
46-
const root = String(input?.workspaceRoot || '').trim();
58+
const root = resolveWorkspaceRoot(input);
4759
if (!rel) return { ok: false, path: rel, error: 'path required' };
4860
if (!root) return { ok: false, path: rel, error: 'workspaceRoot required' };
4961
const abs = resolveUnderRoot(root, rel);
@@ -71,6 +83,8 @@ async function readFileBounded(input: {
7183

7284
async function listDirBounded(input: {
7385
workspaceRoot?: string;
86+
workspacePath?: string;
87+
root?: string;
7488
path?: string;
7589
maxEntries?: number;
7690
}): Promise<{
@@ -81,7 +95,7 @@ async function listDirBounded(input: {
8195
error?: string;
8296
}> {
8397
const rel = String(input?.path || '.').trim() || '.';
84-
const root = String(input?.workspaceRoot || '').trim();
98+
const root = resolveWorkspaceRoot(input);
8599
if (!root) return { ok: false, path: rel, error: 'workspaceRoot required' };
86100
const abs = resolveUnderRoot(root, rel === '.' ? '' : rel);
87101
if (!abs) return { ok: false, path: rel, error: 'path escapes workspaceRoot' };
@@ -150,6 +164,8 @@ const FORBIDDEN_GIT_SUB = new Set([
150164

151165
async function runCommandBounded(input: {
152166
workspaceRoot?: string;
167+
workspacePath?: string;
168+
root?: string;
153169
command?: string;
154170
args?: string[];
155171
timeoutMs?: number;
@@ -163,7 +179,7 @@ async function runCommandBounded(input: {
163179
stderr: string;
164180
error?: string;
165181
}> {
166-
const root = String(input?.workspaceRoot || '').trim();
182+
const root = resolveWorkspaceRoot(input);
167183
const command = String(input?.command || '').trim();
168184
const args = Array.isArray(input?.args)
169185
? input!.args.map((a) => String(a))

packages/pipeline-hosts/src/dev/run-local-deposit-pipeline-debug.ts

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -146,17 +146,17 @@ const debugEnv: Record<string, string> = {
146146
process.env.BITCODE_DEBUG_FORCE_CLONE_PTRR || '0',
147147
BITCODE_DEBUG_STOP_AFTER_FIRST_REASON: '1',
148148
BITCODE_DEBUG_STOP_PHASE: process.env.BITCODE_DEBUG_STOP_PHASE || 'discovery',
149-
// After Plan step closed (1.D14–1.D22): default next is Try PCC reason.
150-
BITCODE_DEBUG_STOP_STEP: process.env.BITCODE_DEBUG_STOP_STEP || 'try',
149+
// After codebase agent closed: next wave-1 = inherent regurgitation Plan PCC.
150+
BITCODE_DEBUG_STOP_STEP: process.env.BITCODE_DEBUG_STOP_STEP || 'plan',
151151
BITCODE_DEBUG_STOP_FAILSAFE:
152152
process.env.BITCODE_DEBUG_STOP_FAILSAFE || 'prepare_concise_context',
153153
BITCODE_DEBUG_STOP_GENERATION:
154154
process.env.BITCODE_DEBUG_STOP_GENERATION || 'reason',
155155
BITCODE_DEBUG_STOP_REQUIRE_GEN0:
156156
process.env.BITCODE_DEBUG_STOP_REQUIRE_GEN0 || '0',
157-
// First Discovery LLM proves Setup (MCP + danger-wall) completed.
158157
BITCODE_DEBUG_STOP_AGENT_FILTER:
159-
process.env.BITCODE_DEBUG_STOP_AGENT_FILTER || 'DepositCodebaseComprehension',
158+
process.env.BITCODE_DEBUG_STOP_AGENT_FILTER ||
159+
'DepositInherentRegurgitation',
160160
BITCODE_PIPELINE_HOST_MAX_RUNTIME_MS:
161161
process.env.BITCODE_PIPELINE_HOST_MAX_RUNTIME_MS || '900000',
162162
// 0 = disable per-call bound for progressive QA (Judge can exceed 180s)

0 commit comments

Comments
 (0)