Skip to content

[Fix] Patch coverage drops despite dedicated lane tests - #1650

Merged
edelauna merged 1 commit into
mainfrom
fix/union-extension-coverage-0crc5m9higm19
Sep 16, 2026
Merged

edelauna merged 1 commit into
mainfrom
fix/union-extension-coverage-0crc5m9higm19

Conversation

@zoomote

@zoomote zoomote Bot commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

​Created by Roomote. Follow up by mentioning @roomote, in the web UI, or in Discord.

What changed

  • Union the five extension ownership-lane LCOV reports before the existing ubuntu Codecov upload.
  • Treat a line, branch, or function as covered when any lane executes it while retaining zero coverage when every lane misses it.
  • Verify the merged tracefile in CI and retain both the source lane reports and merged report as artifacts.
  • Add focused regression coverage for overlapping and disjoint LCOV records.

Why this change was made

Commit 99025b1fb replaced one monolithic extension report with five reports that each instrument the full src source population. Sending those overlapping records together in one Codecov upload caused the default patch status to reflect a sparse lane instead of the union, producing the confirmed false red on #1505.

Closes #1647.

Impact

Patch coverage now reflects execution across all extension test lanes without adding new long-lived flags or carryforward behavior. The existing webview-ui, core-unit, and core-integration uploads and flag semantics remain unchanged.

@coderabbitai

coderabbitai Bot commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

Review Change StackReview Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 58996bfc-6dc2-4157-9fa0-bd0d31ad9bfb

📥 Commits

Reviewing files that changed from the base of the PR and between 8c96629 and 11c97b8.

📒 Files selected for processing (4)
  • .github/workflows/code-qa.yml
  • src/package.json
  • src/scripts/__tests__/merge-lcov.spec.mjs
  • src/scripts/merge-lcov.mjs

Included review availability: Your plan provides up to 4 included reviews per hour; 1 remains after this review.

📜 Recent review details
⏰ Context from checks skipped due to timeout. (2)
  • GitHub Check: e2e-mock
  • GitHub Check: platform-unit-test (windows-latest)
🧰 Additional context used
📓 Path-based instructions (5)
Require regression coverage at the lowest valid harness with behavior-focused assertions, including relevant negative, error, false/unset, and boundary cases.

⚙️ CodeRabbit configuration file

Files:

  • src/scripts/__tests__/merge-lcov.spec.mjs
Check strict typing and exhaustive behavior across normal, boundary, error, cancellation, retry, and compatibility paths.

⚙️ CodeRabbit configuration file

Files:

  • src/scripts/__tests__/merge-lcov.spec.mjs
  • src/scripts/merge-lcov.mjs
Require full commit SHA pins, least-privilege permissions, safe expression and shell interpolation, and trusted metadata handling.

⚙️ CodeRabbit configuration file

Files:

  • .github/workflows/code-qa.yml
Verify extension/webview contracts, cancellation and error propagation, VS Code lifecycle correctness, and behavior under retries and partial failure.

⚙️ CodeRabbit configuration file

Files:

  • src/package.json
  • src/scripts/__tests__/merge-lcov.spec.mjs
  • src/scripts/merge-lcov.mjs
Act as an adversarial second-opinion reviewer.

⚙️ CodeRabbit configuration file

Files:

  • src/package.json
  • src/scripts/__tests__/merge-lcov.spec.mjs
  • src/scripts/merge-lcov.mjs
🔇 Additional comments (2)
src/package.json (1)

446-446: LGTM!

.github/workflows/code-qa.yml (1)

176-180: LGTM!

Also applies to: 192-194, 201-201, 243-243


📝 Summary

Summary by CodeRabbit

  • Chores

    • Consolidated coverage results from all test lanes into a single report for Codecov and downloadable CI artifacts.
    • Coverage reports are now consistently validated and combined before publication.
  • Tests

    • Added automated checks for combined coverage across executed lanes, uncovered lines, and separate source files.
    • Added validation for malformed or conflicting coverage data to improve reporting reliability.

Walkthrough

The PR adds an LCOV merge utility, tests its coverage union behavior, wires it into the package scripts, and updates CI to verify, upload, and archive the merged extension coverage report.

Changes

Coverage report union

Layer / File(s) Summary
LCOV merge engine
src/scripts/merge-lcov.mjs, src/scripts/__tests__/merge-lcov.spec.mjs
The new utility validates LCOV records, merges counts by source path, preserves uncovered lines, and emits sorted records. Tests cover functions, branches, lines, totals, and separate source paths.
Package command wiring
src/package.json
The merge:coverage script runs the merge utility with the merged and per-lane LCOV reports.
CI merge and upload flow
.github/workflows/code-qa.yml
The workflow generates and verifies the merged report, uploads it to Codecov, and adds it to the coverage artifact.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Bug fix · Severity of issue fixed: Medium

Suggested reviewers: edelauna

Sequence Diagram(s)

sequenceDiagram
  participant UnitTestJob
  participant mergeCoverage
  participant Codecov
  participant CoverageArtifact
  UnitTestJob->>mergeCoverage: run merge:coverage
  mergeCoverage->>UnitTestJob: write merged lcov.info
  UnitTestJob->>Codecov: upload merged lcov.info
  UnitTestJob->>CoverageArtifact: store merged lcov.info
Loading

Merge Risk: ⚪ Minimal · up to 11c97

The extension coverage reports are merged and uploaded through consistent CI paths, with no remaining actionable merge risk.


Caution

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

  • Ignore (reviewers only)

❌ Failed checks (1 error, 1 warning)

Check name Status Explanation Resolution
Persistence Integrity ❌ Error The new merge command writes the persisted report non-atomically. src/scripts/merge-lcov.mjs:134 calls writeFileSync directly on src/coverage/merged/lcov.info, which truncates the destination be… Generate the merged LCOV content first, write it to a uniquely named temporary file in src/coverage/merged, and rename it to lcov.info only after the write succeeds. Remove the temporary file on write failure. Use the same atomic replac…
Regression Evidence ⚠️ Warning The new merge utility has explicit error and negative behavior without focused tests. src/scripts/merge-lcov.mjs throws for unfinished, duplicate, conflicting, invalid-count, unsupported, and out-of… Add focused unit tests for representative malformed and conflicting LCOV inputs, including invalid counts, duplicate or conflicting records, unsupported data, and missing CLI arguments. Add assertions that branches and functions remain unco…
✅ Passed checks (6 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Issue #1647 requires union coverage across the five extension lanes, preservation of uncovered lines, and unchanged webview-ui, core-unit, and core-integration flags. merge-lcov.mjs merges line, bra…
Out of Scope Changes check ✅ Passed The changed workflow steps, merge:coverage script, LCOV merger, and focused tests directly implement Issue #1647. No unrelated product behavior or unrelated configuration change is shown in the avai…
Security Boundaries ✅ Passed No changed path meets the failure condition. The new merge-lcov.mjs reads fixed coverage files, parses data into Map objects, validates counts and record structure, and writes only the fixed `cove…
Lifecycle Resource Cleanup ✅ Passed No changed lifecycle path introduces a persistent resource or duplicate work after cancellation, disposal, or restart. The new merge-lcov.mjs path uses synchronous readFileSync and writeFileSync
Title check ✅ Passed The title clearly identifies the patch coverage regression and the dedicated test lanes involved.
Description check ✅ Passed The description explains the cause, implementation, expected behavior, impact, linked issue, and regression coverage. It does not include the template's formal Test Procedure or completed checklist, b…
Full details: Regression Evidence

Explanation

The new merge utility has explicit error and negative behavior without focused tests. src/scripts/merge-lcov.mjs throws for unfinished, duplicate, conflicting, invalid-count, unsupported, and out-of-record LCOV data (lines 19-71), for cross-report function/checksum conflicts (88-96), and for missing CLI arguments (131-134). The new suite only tests valid merges, source-path preservation, and an uncovered DA line. It does not test any rejection path, all-lane-uncovered branches or functions, or the CLI entry point. The CI workflow now invokes this utility during the extension coverage merge, so these are changed behaviors with plausible regression scenarios.

Resolution

Add focused unit tests for representative malformed and conflicting LCOV inputs, including invalid counts, duplicate or conflicting records, unsupported data, and missing CLI arguments. Add assertions that branches and functions remain uncovered when every lane reports zero hits. Add a small CLI test that verifies successful output creation and usage failure.

Full details: Persistence Integrity

Explanation

The new merge command writes the persisted report non-atomically. src/scripts/merge-lcov.mjs:134 calls writeFileSync directly on src/coverage/merged/lcov.info, which truncates the destination before the complete LCOV content is written. If the process or runner stops during the write, the merged report can be truncated and the previous report, if present, is lost. The workflow then uses this path for the Codecov upload at .github/workflows/code-qa.yml:200-204 and the artifact at :237-243. The change adds no temporary-file, atomic-rename, or cleanup path for partial writes.

Resolution

Generate the merged LCOV content first, write it to a uniquely named temporary file in src/coverage/merged, and rename it to lcov.info only after the write succeeds. Remove the temporary file on write failure. Use the same atomic replacement path for reruns so an interrupted write cannot replace a valid report with a partial report.

  • Fix all pre-merge checks with AI
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/union-extension-coverage-0crc5m9higm19

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

Review status

This PR was opened by an automated account. A human maintainer must verify the change intent, provenance, and validation before merging.

Current step: Wait for required CI checks; awaiting-maintainer requires CI and automated review completion.

Review-state labels are managed by this workflow; do not edit them manually.

@codecov

codecov Bot commented Sep 16, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@zoomote

zoomote Bot commented Sep 16, 2026

Copy link
Copy Markdown
Contributor Author

The Persistence Integrity finding does not apply to this generated CI artifact, so no code change was made.

src/coverage/merged/lcov.info is produced in an ephemeral runner workspace and is neither durable state nor an input reused across runs. The merge uses synchronous writeFileSync; if it throws, the step fails. If the process or runner is interrupted, the step never reports success. GitHub Actions applies an implicit success() condition to later steps, so LCOV verification, Turbo cache save, Codecov upload, and artifact upload do not run after a failed or cancelled merge step. A rerun starts from a fresh checkout and regenerates the report.

The hosted run confirms the intended sequence completed successfully on 11c97b8f0: both platform unit-test jobs passed, codecov/patch passed, and Codecov reports all modified coverable lines covered. Atomic replacement would add temporary-file naming, cleanup, and cross-platform rename behavior without protecting durable user data or enabling a downstream consumer to observe a partial file.

No review thread exists to resolve; the error-level pre-merge judgment requires reviewer override or reevaluation.

@edelauna
edelauna marked this pull request as ready for review September 16, 2026 00:48
@edelauna
edelauna enabled auto-merge September 16, 2026 00:48
@edelauna
edelauna added this pull request to the merge queue Sep 16, 2026
Merged via the queue into main with commit fdef106 Sep 16, 2026
30 of 31 checks passed
@edelauna
edelauna deleted the fix/union-extension-coverage-0crc5m9higm19 branch September 16, 2026 01:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] codecov/patch reports far below its 80% target on a diff whose changed files carry dedicated lane tests

2 participants