dream(generator-genome): #248 close zero-test-coverage gap in genome-scorers.ts (evaluated) - #249
Draft
ruvnet wants to merge 2 commits into
Draft
dream(generator-genome): #248 close zero-test-coverage gap in genome-scorers.ts (evaluated)#249ruvnet wants to merge 2 commits into
ruvnet wants to merge 2 commits into
Conversation
…scorers.ts (evaluated) genome-scorers.ts's 5 pure, deterministic, exported functions feed genome.ts's user-visible ready/needs-work/blocked verdict and process exit code, but had zero test coverage anywhere in the package. Adds a 33-test characterization suite with zero production code changes. Baseline: 458 passed, 2 skipped (40 files). Candidate: 491 passed, 2 skipped (41 files) -- 0 regressions. Each new test proven non-vacuous via seeded single-line mutation (5 mutants, one per function, all caught, source restored byte-clean). Independent critic subagent re-verified with its own 6th mutant and returned ACCEPT-WITH-CAVEATS. Research independently surfaced that scoreMcpRisk's local_permissive branch is currently unreachable via the real CLI path (recommendPlan() always assigns a hardcoded SAFE policy) -- now documented explicitly in the new suite rather than left as a silent gap. Deliberately does not fix the two known bugs in this file that already have unreviewed draft PRs open (#200 scorePublishReadiness language credit, #229 scoreTestConfidence file verification) -- this suite instead documents their current buggy behavior with tests named "documented gap", so those PRs land cleanly without a 3rd competing fix deepening the review backlog. See docs/dream-cycle/2026-08-30-gist.md for full research, competitor comparison, and evaluation receipt. Human review required. Does not self-merge or autonomously promote Flywheel state. Co-Authored-By: Claude <noreply@anthropic.com>
main's LEDGER.md was stuck at 2 rows (08-13, 08-14) while 11 more real Dream Cycle nights ran and pushed branches/PRs -- STEP 25 silently failed 08-15..08-26 and its own backfill (#240) is unmerged, so main never got the correction. Adds a summary backfill row for 08-15..08-29 (0/11 merged, 2 disclosed unpatched security fixes) plus tonight's own 2026-08-30 row (issue #248, PR #249). Co-Authored-By: Claude <noreply@anthropic.com>
This was referenced Aug 31, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hypothesis
Given
genome-scorers.ts's 5 pure, deterministic, exported functions (classifyRepoType,resolveAgentTopology,scoreMcpRisk,scoreTestConfidence,scorePublishReadiness), which feedgenome.ts's user-visibleready/needs-work/blockedverdict and process exit code, when a dedicated vitest suite is added covering boundary/representative behavior, then the surface gains characterized regression coverage (0→33 tests) with zero production diff, subject to: byte-identicalgenome-scorers.ts, 0 regressions in the existing 458-test suite, and every new test provably non-vacuous via seeded mutation.Candidate
One new file:
packages/create-agent-harness/__tests__/genome-scorers.test.ts(33 tests). Zero changes to any production source file.Evaluation Receipt
genome-scorers.ts— 0 dedicated tests. Full package suite: 458 passed, 2 skipped (40 files).tsc --noEmit: clean, 0 errors.git diffconfirmed byte-clean after every revert.evaluated: acceptedBaseline Comparison
458 → 491 tests, 0 production diff, 0 regressions, 0 skipped-test change.
Darwin Lineage
Not run — deterministic discrete-branch scorer, no continuous parameter space to explore.
Flywheel Evidence
No flywheel gate/promotion receipt applies (test-only, non-LLM-call candidate). This PR + the linked issue + the gist constitute the evidence record.
Reward Hack Check
Checked by both the candidate author and an independent critic subagent (separate context): no test weakening, no gold/corpus modification, no hardcoded outputs, no
.skip/.only, no metric substitution, no hidden preprocessing. Clear.Security Review
Not security-sensitive (test-only, no new runtime code path). One security-adjacent observation carried forward for a future night (not fixed here, to keep this patch single-file and zero-production-diff):
hasCi/hasMcpinanalyze-repo.tsare loose regex text-matches over README/manifest prose rather than structural checks.Regression Analysis
0 regressions (458→491, additive only). Two tests are deliberately named
documented gapand assert the current, known-buggy values ofscoreTestConfidence/scorePublishReadiness— these are intentional characterization of today's behavior, cross-referenced to the two already-open draft fix PRs below, not something this PR is trying to hide.ADR
None — this is test coverage, not an architectural decision (no ADR created per the "never for parameter changes, benchmark additions, documentation" rule).
Research Gist
docs/dream-cycle/2026-08-30-gist.md(this branch). Full sourced 2026 research, competitor comparison (OpenSSF Scorecard, AgentLint, agnix, LangGraph/AutoGen/CrewAI/OpenAI Agents SDK/Claude Agent SDK, MCP-Scan), and scan findings forrouter/turn-credit. A public gist was not created —ghCLI is unavailable in this session's environment.Issue
#248
Witness
Operationally more urgent than this PR
0 of ~13 Dream Cycle PRs since 2026-08-14 have merged, including two disclosed RCE-class security fixes still unpatched on
main: #212 (host-adapters codegen shell/YAML injection, open 40+ days) and #224 (host-rvm shell RCE, open ~2 weeks). See issue #248's "Ledger Check" section for the full verified backlog state. Please triage that backlog before or alongside reviewing this PR — this PR itself is intentionally low-risk (single file, zero production diff) and can wait.Merge Policy
Human review required. Do not self-merge. Do not autonomously promote Flywheel state.
Generated by Claude Code