Skip to content

chore(test): coverage tooling + test-coverage analysis - #373

Open
mavaali wants to merge 1 commit into
mainfrom
claude/test-coverage-analysis-49d1gb
Open

chore(test): coverage tooling + test-coverage analysis#373
mavaali wants to merge 1 commit into
mainfrom
claude/test-coverage-analysis-49d1gb

Conversation

@mavaali

@mavaali mavaali commented Aug 12, 2026

Copy link
Copy Markdown
Owner

Summary

Adds @vitest/coverage-v8 as a devDependency, a test:coverage script (vitest run --coverage --coverage.include='src/**/*.ts' --coverage.reportOnFailure), and a .gitignore entry for the generated coverage/ directory. This came out of a test-coverage analysis of the codebase (requested via task); the measured baseline and proposed focus areas are below so they're recorded with the tooling that produces them.

Measured baseline (main src/ tree only, 2,819 tests)

89.2% line / 77.2% branch coverage overall. Leaf/domain logic is very well covered (fence, canary, hooks, witness at 100%; curation 94.6%, tools 92.3%, search 92.0%, storage 93.7%). The gaps concentrate in orchestration/entrypoint layers:

Area Lines Notes
src/index.ts 27% stdio entry — only exercised e2e via built dist/, which v8 coverage doesn't attribute
src/eval/llm.ts 46% Anthropic client: retry loop, tool-round loop, error mapping untested
src/cli.ts 47% subcommand dispatch branches
src/import/langgraph-store.ts 50% (38% branch) foreign-store mapping edge cases
src/sleep/ 74% report.ts 54%, index.ts 59% — cycle orchestration
src/serve/index.ts 72% strong auth/gating tests exist; gap is shutdown/signal/sync-loop error paths
src/access/locks.ts 74% (63% branch) security-relevant, small — cheap to close
src/storage/backend.ts 88% (52% branch) s3/azure error-mapping branches

Proposed focus areas

  1. Wire test:coverage into CI with a threshold (~85% lines) so the baseline can't erode silently — no workflow currently reports coverage.
  2. Finish src/access/locks.ts branches (contested acquire, expired-lock takeover, non-holder release) — highest value per line.
  3. Test eval/llm.ts retry/error/tool-loop against a stubbed SDK client.
  4. Extract the arg-parsing/startup gating in src/index.ts and src/cli.ts into pure functions and unit-test them (the validateServeStartup pattern already used in serve is the precedent).
  5. Table-driven tests for import/langgraph-store.ts branch matrix.
  6. Guard environment-sensitive tests: 22 files fail in a sandbox without network (MiniLM download), without a prior npm run build (dist/-dependent e2e), or when run as root (chmod-unreadable fixtures) — describe.skipIf guards like the ones in audit.perf / eval smoke would make the suite green anywhere.

Test plan

  • npm run build passes locally
  • npm test — 2,774 passed; the 31 failures are sandbox-environment-only (no network for the HF model download, no dist/ at first run, root user defeats chmod fixtures) and reproduce on main
  • New behavior has tests (tests mirror src/; every tool gets a test file) — n/a: tooling-only change, no src/ behavior touched

Invariants

No src/ code touched — tooling/config only, so the CLAUDE.md invariants are unaffected.

🤖 Generated with Claude Code

https://claude.ai/code/session_01L6TubGkW4b529nMAJNbiRX


Generated by Claude Code

Adds @vitest/coverage-v8 and a test:coverage script so line/branch
coverage can be measured locally and later wired into CI. Ignores the
generated coverage/ output.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L6TubGkW4b529nMAJNbiRX
@claude claude Bot mentioned this pull request Aug 12, 2026
@mavaali
mavaali marked this pull request as ready for review August 14, 2026 00:24
@claude

claude Bot commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Review summary

Reviewed for correctness bugs, simplification/efficiency, and CLAUDE.md invariants (frontmatter-as-metadata, derived SQLite index, git-as-version-layer, advisory-only curation, existence-disclosure rules, Result<T, Error>, no classes, test-file-per-tool).

No findings. This PR only touches .gitignore, package.json, and package-lock.json — no src/ code is touched, so none of the CLAUDE.md invariants are implicated and the "every tool needs a test file" rule doesn't apply (no new tool). Specifically checked and confirmed clean:

  • @vitest/coverage-v8@4.1.10's peer dependency requires vitest@4.1.10 exactly, and the lockfile resolves the project's vitest (^4.1.6) to 4.1.10 — no version mismatch.
  • The new test:coverage script isn't wired into CI in this PR, matching the PR description's framing of that as a follow-up ("Proposed focus area Conflict resolution beyond file-level write locks #1"), not this change.
  • The peerDependencies reordering in package.json (pg moved after aws-sdk/azure) is cosmetic only, functionally equivalent.
  • .gitignore addition of coverage/ is correct and matches the new script's output directory.

Nothing to flag.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants