Skip to content

[Dream Cycle 2026-09-10] compiler-parity: validateConfig() accepts blank/whitespace-only slot scan entries and deep field + config-schema,golden-snapshots scan #104

Description

@ruvnet

Rotation

DATE=2026-09-10, SLOT=0DEEP=compiler-parity, SCAN=config-schema,golden-snapshots. Session commit 7933c3599abe22df5290f4609d1f93f598feb3de (main tip at session start).

Ledger Check

docs/dream-cycle/LEDGER.md on main carries only 6 rows (last: 2026-08-26) — stale relative to the ~40-row accumulated ledger sitting across unmerged dream/* branches (confirmed zeroMergeStreak-adjacent: PR #24 the lone self-hosting-relevant merge, 2026-08-26). Fetched all origin/dream/* branches to inspect prior nights' actual (unmerged) work rather than rely on main's stale ledger alone, per STEP 2's "load accumulated evidence" — gh CLI is unavailable this session but the GitHub MCP server provided equivalent read access (issues/PRs), so this is not a FALLBACK=true night for GitHub state, only for gist publication (no gist-creation tool available).

Three prior compiler-parity nights on file: 2026-08-15 (missing test coverage → golden snapshot, PR #11), 2026-08-25 (adrConvention shape validation, issue #28/PR #29, still open), 2026-09-05 (bonusModuli value validation, issue #78/PR #79, still open). This finding is a fourth, scoped to a different, still-open gap in the same validateConfig() function — not a duplicate of either open PR.

Deep Dive

packages/compile/src/config.ts's validateConfig() validates slots[].deep truthiness (!s.deep) and slots[].scan array length (s.scan.length < 1), but never validates that individual scan array entries, or deep itself, are non-blank after trimming. A slot with scan: ['', 'tests'] or deep: ' ' passes validation cleanly (ok: true, no errors) and compile() then silently emits a corrupted line: SCAN=,tests (dangling comma, blank surface) or DEEP= (whitespace-only, no visible surface name). Live pre-fix reproduction:

$ node -e "... cfg.slots[0].scan = ['', 'tests']; validateConfig(cfg) ..."
{"ok":true,"errors":[],"warnings":[]}
compiled SCAN line -> "   SCAN=,tests"

This is the same defect class as three prior ACCEPTed/proposed compiler-parity fixes (cron minute field, PR #24 merged; adrConvention shape, issue #28/PR #29; bonusModuli values, issue #78/PR #79): a config field that's technically present and passes the existing schema check, but degenerate enough to silently corrupt the compiled routine prompt instead of failing loud.

Hypothesis

Given a dream.config whose slots[i].scan array contains an empty-string or whitespace-only entry, or whose slots[i].deep is whitespace-only, extending validateConfig() to reject blank surface names in both fields should make validateConfig report ok: false for such configs, with compile() never reachable with a blank SCAN=/DEEP= line for any config that passes validation — subject to 0 regressions and no change to any currently-valid config's result (self-hosted dream.config.json and the metaharness fixture both byte-identical before/after).

Evaluation Receipt

Real evaluator: npm test (vitest). Baseline 7933c359 (session-start main): 98/98 green. Candidate: 101/101 green (+3, 0 removed/modified). Build clean (6 packages, no wasm/NAPI degradation). Self-hosted dream.config.json compile output confirmed byte-identical before/after (diff clean — it was already well-formed, so no behavior change for the real config; the fix only changes behavior for configs the schema should have rejected).

Darwin Results

DARWIN=not-applicable. Single deterministic per-field blank-string check added to an existing validation loop — no evolvable population, same judgment as every prior single-function compiler-parity/developer-experience fix in this ledger.

Evidence

OBSERVATION (grade A, live pre-fix reproduction) → OBSERVATION (grade A, code read confirms no other branch/PR already covers this exact gap — checked all 28 fetched origin/dream/* branches' config.ts) → MEASUREMENT (grade A, 98→101 tests, 0 regressions) → MEASUREMENT (grade A, self-hosted config byte-identical) → INFERENCE (independent critic subagent, fresh context, confirmed bug real by stashing the fix and re-testing) → DECISION (scope fix to deep/scan blank-string checks only, not the critic-flagged residual gap in labels/ledgerPath/competitors/etc. — avoiding unrequested scope creep) → REJECTION (did not resubmit adrConvention (#28/#29) or bonusModuli (#78/#79), both already open).

Reward-Hack Check

Independent critic (fresh subagent, no shared authoring context, git diff + full repo read access): CLEAR. Confirmed the bug is real by stashing only config.ts and re-testing against the pre-fix version. Confirmed the two new tests are non-vacuous — both fail against the stashed pre-fix code and pass post-fix. No .only/.skip, no snapshot/gold-data edits, no threshold changes, no undocumented cache. Confirmed no regression: metaharness fixture and self-hosted dream.config.json both still validate cleanly post-fix (tightened check rejects nothing currently valid). One residual-scope note from the critic: other interpolated DreamConfig string fields (labels, competitors, ledgerPath, branchPrefix, evaluatorEntrypoints.*, controlPlaneProbes, extraDisciplines) remain unvalidated for blank/whitespace entries — correctly out of scope for tonight's narrow candidate, flagged here as a legitimate next-night direction rather than fixed opportunistically.

Security Review

Pure string-validation tightening inside an existing validator — no new I/O, no LLM/prompt-injection surface, no MCP/tool-authority change, no credential exposure, no filesystem/network-scope change. Strictly narrows what previously validated ok: true; cannot newly accept anything that was previously rejected.

Scan Findings

config-schema: confirmed the exact gap above by walking every DreamConfig field against what validateConfig checks vs. what compile()/index.ts interpolates unescaped into the template — slots[].deep/slots[].scan[] was the one remaining unchecked-for-blankness field group not already covered by an open PR (adrConvention, bonusModuli values, cron minute field are the other three, already found on prior nights).

golden-snapshots: packages/compile/src/index.test.ts already carries a metaharness golden-snapshot test (golden-snapshot: metaharness prompt is stable) on main — pre-existing coverage from an earlier accepted night, unaffected by tonight's change (self-hosted dream.config.json compile output confirmed byte-identical, so no snapshot drift).

Competitors

See the attached report (docs/dream-cycle/2026-09-10-compiler-parity-report.md) for the full competitor table (Sakana AI Scientist, OpenHands, DSPy/GEPA, SWE-agent, AutoGPT lineage) — all graded B/C (no A-grade external evidence found for this exact defect class); the ACCEPT verdict rests on first-hand reproduction against this repo's own code, not the competitor survey.

Gist

No gh CLI or gist-creation MCP tool available this session (consistent with every prior night per the accumulated ledger). GIST=LOCAL. Full report committed at docs/dream-cycle/2026-09-10-compiler-parity-report.md.

Witness

report_sha256 : 196cd963b339a38e4cb2bffd697ef2967c1e931fc7a8f670e980066c4290e37b
session_commit: 7933c3599abe22df5290f4609d1f93f598feb3de
witness       : 2eae81eb6c138360f60e395e06d33cce59cc4cfe5f511c91f4878601505d3036

Computed via node packages/cli/dist/bin.js witness stamp <report> <commit> over the report's pre-Witness-section content (see the report's own Witness section for the full 5-step reproduce-it chain).

Recommendation

evaluated: accepted. The PR opened against this issue does not self-merge and does not carry the automerge-safe label — human review required, same as every prior night.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions