Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
276 changes: 276 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,13 @@ permissions:
jobs:
test:
runs-on: ubuntu-latest
env:
HEAD_SHA: ${{ github.event.pull_request.head.sha || github.sha }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
ref: ${{ github.event.pull_request.head.sha || github.sha }}
- name: Install Tauri system deps
run: |
sudo apt-get update
Expand All @@ -40,14 +43,287 @@ jobs:
node-version: 20.19.0
- run: npm ci
- run: npm test
- name: Frontend production coverage
id: frontend-coverage
run: npm run coverage
- name: Build bounded frontend coverage diagnostic
if: failure() && steps.frontend-coverage.outcome == 'failure'
run: |
node --input-type=module <<'NODE'
import { appendFileSync, readFileSync, writeFileSync } from 'node:fs';

const sha = process.env.HEAD_SHA ?? '';
const repository = process.env.GITHUB_REPOSITORY ?? '';
if (!/^[0-9a-f]{40}$/.test(sha) || repository.length === 0) {
throw new Error('frontend coverage diagnostic identity is invalid');
}

const finalCoverage = JSON.parse(
readFileSync('coverage/coverage-final.json', 'utf8'),
);
const summaryCoverage = JSON.parse(
readFileSync('coverage/coverage-summary.json', 'utf8'),
);
const frontendMarker = '/src/';
const frontendRepositoryPath = (filename) => {
const normalized = String(filename ?? '').replaceAll('\\', '/');
if (normalized.startsWith('src/')) return normalized;
const markerIndex = normalized.lastIndexOf(frontendMarker);
if (markerIndex < 0) return null;
return `src/${normalized.slice(markerIndex + frontendMarker.length)}`;
};
const uncoveredCount = (values) => values.filter((value) => value === 0).length;
const frontend_uncovered_line_numbers = (file) => {
const lines = new Set();
for (const [statementId, count] of Object.entries(file?.s ?? {})) {
if (count !== 0) continue;
const line = file?.statementMap?.[statementId]?.start?.line;
if (Number.isSafeInteger(line) && line > 0) lines.add(line);
}
return [...lines].sort((left, right) => left - right).slice(0, 40);
};
const frontend_top_uncovered_files = Object.entries(finalCoverage)
.map(([filename, file]) => {
const path = frontendRepositoryPath(filename);
if (!path || !file || typeof file !== 'object') return null;
const statements = Object.values(file.s ?? {});
const functions = Object.values(file.f ?? {});
const branches = Object.values(file.b ?? {}).flatMap((value) =>
Array.isArray(value) ? value : [],
);
return {
path,
uncovered_statements: uncoveredCount(statements),
uncovered_branches: uncoveredCount(branches),
uncovered_functions: uncoveredCount(functions),
uncovered_line_numbers: frontend_uncovered_line_numbers(file),
};
})
.filter((entry) => entry && (
entry.uncovered_statements > 0 ||
entry.uncovered_branches > 0 ||
entry.uncovered_functions > 0 ||
entry.uncovered_line_numbers.length > 0
))
.sort((left, right) => {
const leftGap = left.uncovered_statements + left.uncovered_branches +
left.uncovered_functions + left.uncovered_line_numbers.length;
const rightGap = right.uncovered_statements + right.uncovered_branches +
right.uncovered_functions + right.uncovered_line_numbers.length;
return rightGap - leftGap || left.path.localeCompare(right.path);
})
.slice(0, 20);
const diagnostic = {
schema_version: 1,
head_sha: sha,
repository,
totals: summaryCoverage?.total ?? null,
frontend_top_uncovered_files,
};
writeFileSync(
'frontend-coverage-diagnostic.json',
`${JSON.stringify(diagnostic, null, 2)}\n`,
);
console.error(`frontend-coverage-diagnostic=${JSON.stringify(diagnostic)}`);
const summaryPath = process.env.GITHUB_STEP_SUMMARY;
if (summaryPath) {
appendFileSync(
summaryPath,
`### Frontend coverage diagnostic for \`${sha}\`\n\n\`\`\`json\n${JSON.stringify(diagnostic, null, 2)}\n\`\`\`\n`,
);
}
NODE
- name: Upload bounded frontend coverage diagnostic
if: failure() && steps.frontend-coverage.outcome == 'failure'
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: frontend-coverage-diagnostic-${{ env.HEAD_SHA }}
path: frontend-coverage-diagnostic.json
if-no-files-found: error
- run: npm run check
- run: npm run build

coverage-evidence:
runs-on: ubuntu-latest
env:
HEAD_SHA: ${{ github.event.pull_request.head.sha || github.sha }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
ref: ${{ github.event.pull_request.head.sha || github.sha }}
- name: Install Tauri system deps
run: |
sudo apt-get update
sudo apt-get install -y libwebkit2gtk-4.1-dev libgtk-3-dev libayatana-appindicator3-dev librsvg2-dev
- uses: dtolnay/rust-toolchain@4be7066ada62dd38de10e7b70166bc74ed198c30
with:
toolchain: nightly-2026-08-07
components: llvm-tools-preview
- uses: taiki-e/install-action@6c6fd71fe4fb72c3697d269963d0e15df8adedad
with:
tool: cargo-llvm-cov
- uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32 # v2
with:
workspaces: src-tauri
- name: Measure exact-head Rust coverage
run: cargo llvm-cov --manifest-path src-tauri/Cargo.toml --branch --json --output-path coverage.json --no-cfg-coverage --no-cfg-coverage-nightly
- name: Build exact-head coverage evidence
run: |
node --input-type=module <<'NODE'
import { appendFileSync, readFileSync, writeFileSync } from 'node:fs';

const sha = process.env.HEAD_SHA ?? '';
const repository = process.env.GITHUB_REPOSITORY ?? '';
if (!/^[0-9a-f]{40}$/.test(sha) || repository.length === 0) {
throw new Error('coverage evidence identity is invalid');
}

const report = JSON.parse(readFileSync('coverage.json', 'utf8'));
const coverageData = report?.data?.[0];
const totals = coverageData?.totals;
const safeGap = (value) => {
const count = value?.count;
const covered = value?.covered;
return Number.isSafeInteger(count) &&
Number.isSafeInteger(covered) &&
count >= covered &&
covered >= 0
? count - covered
: 0;
};
const repositoryPath = (filename) => {
const normalized = String(filename ?? '').replaceAll('\\', '/');
const marker = '/src-tauri/';
const markerIndex = normalized.lastIndexOf(marker);
if (markerIndex < 0) return null;
return `src-tauri/${normalized.slice(markerIndex + marker.length)}`;
};
const uncoveredLineNumbers = (segments) => {
const lines = new Set();
for (const segment of Array.isArray(segments) ? segments : []) {
if (
Array.isArray(segment) &&
Number.isSafeInteger(segment[0]) &&
segment[0] > 0 &&
segment[2] === 0 &&
segment[3] === true &&
segment[4] === true &&
segment[5] !== true
) {
lines.add(segment[0]);
}
}
return [...lines].sort((left, right) => left - right).slice(0, 40);
};
const top_uncovered_files = (coverageData?.files ?? [])
.map((file) => {
const path = repositoryPath(file?.filename);
const summary = file?.summary;
if (!path) return null;
return {
path,
uncovered_regions: safeGap(summary?.regions),
uncovered_branches: safeGap(summary?.branches),
uncovered_functions: safeGap(summary?.functions),
uncovered_lines: safeGap(summary?.lines),
uncovered_line_numbers: uncoveredLineNumbers(file?.segments),
};
})
.filter((entry) => entry && (
entry.uncovered_regions > 0 ||
entry.uncovered_branches > 0 ||
entry.uncovered_functions > 0 ||
entry.uncovered_lines > 0
))
.sort((left, right) => {
const leftGap = left.uncovered_regions + left.uncovered_branches +
left.uncovered_functions + left.uncovered_lines;
const rightGap = right.uncovered_regions + right.uncovered_branches +
right.uncovered_functions + right.uncovered_lines;
return rightGap - leftGap || left.path.localeCompare(right.path);
})
.slice(0, 20);
const diagnostic = {
schema_version: 1,
head_sha: sha,
repository,
regions: totals?.regions ?? null,
branches: totals?.branches ?? null,
functions: totals?.functions ?? null,
lines: totals?.lines ?? null,
top_uncovered_files,
};
writeFileSync(
'coverage-diagnostic.json',
`${JSON.stringify(diagnostic, null, 2)}\n`,
);
console.error(`coverage-diagnostic=${JSON.stringify(diagnostic)}`);
const summaryPath = process.env.GITHUB_STEP_SUMMARY;
if (summaryPath) {
appendFileSync(
summaryPath,
`### Coverage diagnostic for \`${sha}\`\n\n\`\`\`json\n${JSON.stringify(diagnostic, null, 2)}\n\`\`\`\n`,
);
}

const metric = (name, value) => {
if (
!value ||
!Number.isFinite(value.count) ||
!Number.isFinite(value.covered) ||
!Number.isFinite(value.percent) ||
value.count <= 0 ||
value.covered !== value.count ||
value.percent !== 100
) {
throw new Error(`${name} coverage is not exactly 100%`);
}
return value.percent;
};

const evidence = {
schema_version: 1,
head_sha: sha,
commit_sha: sha,
repository,
trust_tier: 'ci-verified',
ci_server: 'github-actions',
ci_workflow: 'Test',
coverage_command: 'cargo llvm-cov',
statement_coverage: metric('statement/region', totals?.regions),
branch_coverage: metric('branch', totals?.branches),
function_coverage: metric('function', totals?.functions),
line_coverage: metric('line', totals?.lines),
passed: true,
};

writeFileSync(
'coverage-evidence.json',
`${JSON.stringify(evidence, null, 2)}\n`,
);
NODE
- name: Upload bounded coverage diagnostic
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: coverage-diagnostic-${{ env.HEAD_SHA }}
path: coverage-diagnostic.json
if-no-files-found: error
- name: Upload coverage evidence
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: coverage-evidence
path: coverage-evidence.json
if-no-files-found: error

llm-engine-build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
ref: ${{ github.event.pull_request.head.sha || github.sha }}
- name: Install build deps (llama.cpp native + tauri)
run: |
sudo apt-get update
Expand Down
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and

### Changed

- Measure exact-head Rust production statement-equivalent region, branch, function, and line coverage in CI; require every metric to be exactly 100%, emit privacy-bounded failure diagnostics, and publish success evidence only for the immutable pull-request head that actually passed.
- Replace generator-era Cargo package metadata with the DiskSage product description, MIT license expression, canonical source repository URL, and `publish = false` registry-publication boundary; deliberately omit Cargo's deprecated `authors` field, verify publication refusal through Cargo's versioned parsed metadata rather than substring matching, and regression-test commented/out-of-table decoys together with the retained acquisition metadata and doctoring evidence.
- Require a fresh, exact, human-attributed approval and rationale for cloud copy-only and existing-copy adoption actions, with a 15-minute authorization lifetime bound to the candidate, destination, provider, account scope, and review fingerprint.
- Return the candidate-specific cloud copy approval action, exact confirmation phrase, and maximum approval age from the Rust plan contract; the frontend only displays and submits that backend-authored phrase and fails closed when it is missing or does not match the candidate action.
Expand Down
44 changes: 44 additions & 0 deletions docs/development/coverage-evidence.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# Exact-head coverage evidence

DiskSage treats code coverage as durable CI evidence rather than a locally asserted percentage. The `Test` workflow measures Rust production coverage on the exact pull-request head and emits a machine-readable `coverage-evidence` artifact only when all required metrics are exactly 100%.

## Why the workflow checks out the exact head

GitHub pull-request workflows may otherwise execute against a synthetic merge commit. That is useful for integration testing, but it is not sufficient for a review gate that claims a result about one immutable pull-request head. The coverage job therefore checks out `${{ github.event.pull_request.head.sha || github.sha }}` explicitly and copies the same value into `HEAD_SHA`.

The evidence builder rejects a missing or malformed SHA. Both `head_sha` and `commit_sha` in `coverage-evidence.json` must equal that exact 40-character commit identifier. The repository identity is taken from `GITHUB_REPOSITORY`, not from user-controlled test output.

## What is measured

The workflow uses `cargo llvm-cov` with LLVM source-based instrumentation. Branch coverage is requested explicitly with `--branch`; because cargo-llvm-cov documents branch coverage as unstable, the workflow uses an immutable dated Rust nightly with `llvm-tools-preview` instead of silently falling back to a toolchain that cannot measure the required metric.

The workflow passes `--no-cfg-coverage` and `--no-cfg-coverage-nightly`, so cargo-llvm-cov does not define its normal `cfg(coverage)` or `cfg(coverage_nightly)` build configurations. Production code guarded by `#[cfg(not(coverage))]` therefore remains in the measured graph rather than disappearing merely because coverage is being collected. This keeps the gate aligned with the production-behavior requirement; it also means unreachable GUI or command boundaries must be made realistically testable rather than hidden from measurement.

The JSON summary is the only source for the emitted percentages. The evidence builder reads LLVM's aggregate totals and requires all of the following to be present, finite, non-empty, fully covered, and exactly 100%:

- statement coverage: LLVM region coverage, used as the statement-equivalent source-based metric;
- branch coverage: LLVM branch totals;
- function coverage: LLVM function totals; and
- line coverage: LLVM line totals.

The workflow never manufactures a percentage from a successful test exit status. Missing totals, zero denominators, partial coverage, malformed JSON, or identity drift stop the job before the success artifact can be uploaded.

## Evidence and failure diagnostics

A valid `coverage-evidence.json` has schema version `1` and records the immutable head, repository, CI trust tier, server, workflow name, coverage command, four exact percentages, and `passed: true`. The organization review workflow independently downloads this artifact from the successful `Test` run for the same head and revalidates the contract.

The success artifact is uploaded with `if-no-files-found: error`. GitHub Actions artifacts persist workflow outputs such as test and coverage results after the producing job completes, which lets the organization-level reviewer consume evidence without granting the coverage job repository-write permission.

When any metric is below 100%, the job still writes the bounded `coverage-diagnostic.json` containing only the exact head/repository identity and aggregate region, branch, function, and line totals. The same bounded diagnostic is emitted to the job log and `GITHUB_STEP_SUMMARY` before validation throws, then uploaded with `if: always()`. This makes the first failing coverage boundary directly observable without exposing source contents, local paths, secrets, test fixtures, or command output, while the success-only `coverage-evidence.json` remains fail closed.

## Fail-closed operating rule

A missing `coverage-evidence` artifact is not equivalent to passing coverage. A queued, cancelled, failed, stale-head, malformed, or less-than-100% measurement is also not passing. Engineers must add realistic tests or remove genuinely unreachable production code; they must not lower thresholds, hard-code percentages, exclude reachable production behavior merely to satisfy the gate, or reuse an artifact from an older head.

## References

GitHub. (2026). *Store and share data with workflow artifacts*. GitHub Docs. https://docs.github.com/en/actions/tutorials/store-and-share-data

GitHub. (2026). *Workflow artifacts*. GitHub Docs. https://docs.github.com/en/actions/concepts/workflows-and-actions/workflow-artifacts

Taiki Endo. (2026). *cargo-llvm-cov: Cargo subcommand to easily use LLVM source-based code coverage* [Computer software]. GitHub. https://github.com/taiki-e/cargo-llvm-cov
Loading
Loading