Skip to content

[Quality evidence] Extend non-vacuous 100% coverage to every owned production module #205

Description

@seonghobae

Acquisition / quality-evidence gap

Protected develop@622e5e6c3d534f230c390f10e3832efadfc01825 does not currently enforce repository-wide owned-production coverage. The only JaCoCo plugin configuration is inside etl-service/pom.xml, and its explicit rule is limited to the durable-job/controller selector:

<include>com.xtrmetl.etl.job.*</include>
<include>com.xtrmetl.etl.controller.EtlJobController*</include>

Protected root pom.xml defines all five production modules (etl-service, cdc-service, config-server, eureka-server, zuul-gateway) but has no reactor-wide JaCoCo coverage authority. Protected cdc-service/pom.xml and zuul-gateway/pom.xml likewise contain no JaCoCo coverage gate; current hosted logs show config-server and eureka-server can build with No tests to run.

Issue #162 / PR #164 is necessary but narrower: it repairs the vacuity defect in the selected ETL durable-job bundle and demonstrates a real eight-class bundle under zero-missed instruction/line/method/branch rules. That must not be promoted into a claim that every mightyETL-owned production class across every module has 100% coverage.

Status: known_gap. Source mutation is defer_until_trigger while #164 actively owns the ETL JaCoCo configuration; do not race it or weaken its non-empty/zero-missed invariant.

RCA

  • Immediate cause: coverage enforcement was introduced for one high-risk durable-job slice rather than as a complete reactor product-quality authority.
  • Technical root cause: module-local JaCoCo selection and repository-wide coverage are currently conflated. A green selected bundle says nothing about unselected ETL classes or production classes in CDC, Config Server, Eureka Server, and Gateway.
  • Control failure: no machine-checkable inventory proves that every owned production class belongs to exactly one accepted coverage scope, and no reactor/aggregate gate fails when an owned module contributes no production coverage evidence.
  • Acquisition risk: a buyer can see a strict-looking MISSEDCOUNT <= 0 gate while material production surfaces remain structurally outside the measured set. This is evidence incompleteness, not merely a reporting preference.

Current source-backed evidence

Protected root Maven reactor modules:

  • etl-service;
  • cdc-service;
  • config-server;
  • eureka-server;
  • zuul-gateway.

Protected coverage authority exists only in etl-service/pom.xml and is explicitly scoped to the durable-job/controller classes above. cdc-service/pom.xml and zuul-gateway/pom.xml have only the Spring Boot Maven plugin; the root POM has compiler/Boot/Surefire plugins but no JaCoCo plugin or aggregate coverage gate.

JaCoCo 0.8.15 is the current protected plugin version. JaCoCo's official Maven documentation distinguishes single-project report from multi-project report-aggregate; check evaluates configured elements/counters and therefore cannot prove coverage for classes that were never part of the governed report/check scope.

Target architecture

After #164 stabilizes/integrates, establish one source-backed coverage ownership model for the complete reactor. Prefer a model that makes omissions mechanically impossible rather than maintaining an ever-growing hand-written allowlist.

Materially viable approaches to evaluate against the then-current reactor:

  1. Per-module fail-closed gates + reactor inventory — every production module owns a JaCoCo agent/report/check with zero-missed rules, while a root contract proves every owned source set/module has an enabled non-vacuous gate.
  2. Reactor aggregate coverage authority — collect module execution/class/source data into an aggregate report and enforce exact zero-missed requirements over the complete owned production set, while retaining module-local diagnostics where useful.
  3. Hybrid — per-module non-vacuity plus aggregate repository-level zero-missed acceptance. This is preferred if aggregate tooling alone can hide a missing module or missing execution-data boundary.

Do not select an approach solely because it produces one percentage. The acceptance authority must prove both scope completeness and zero missed owned production behavior.

Fail-closed invariants

The final quality gate must prove at minimum:

  1. every Maven module containing mightyETL-owned production Java belongs to the governed coverage inventory;
  2. every governed module/class set is non-empty before any coverage ratio/count may pass;
  3. every owned production class is included unless a reviewed ADR identifies a generated/external/non-owned exception;
  4. exclusions are explicit, minimal, source-backed, testable, and cannot silently grow;
  5. exact zero missed INSTRUCTION, LINE, METHOD, and BRANCH counters hold for every governed owned production class where JaCoCo exposes the counter;
  6. a newly added production class with no tests makes CI fail rather than disappearing from the report;
  7. a newly added production module with no coverage configuration makes CI fail rather than reporting an unchanged green aggregate;
  8. modules with only startup/bootstrap classes are either realistically tested or explicitly redesigned/documented; No tests to run is not 100% coverage evidence;
  9. unit/integration tests use realistic service boundaries and are not padded with source-text-only assertions solely to hit lines;
  10. conditional/generated/defensive branches remain semantically meaningful—100% structural coverage does not replace domain-validity, security, concurrency, migration, recovery, or interoperability tests;
  11. JaCoCo execution remains compatible with the Mockito startup-agent repair tracked by [Test/Compatibility] Preload Mockito agent instead of dynamic self-attachment #194 and must not lose either agent through argLine replacement;
  12. exact-source identity and scanner-completeness controls (ci: schedule NVIDIA OpenCode maintenance agent #121/[Security evidence] Fail closed when Trivy cannot resolve Maven BOM-managed dependency versions #196) remain separate evidence dimensions and cannot be substituted by coverage green;
  13. canonical Test Strategy, Architecture/quality attributes, Traceability, release acceptance and acquisition documentation state the exact governed coverage scope and evidence identity;
  14. protected-develop operational acceptance proves the integrated gate on the exact protected source before any 100% owned production coverage claim.

TDD / sequencing

When #164 is integrated or otherwise hands off a stable exact coverage boundary:

  1. inventory current production source classes/modules and current test execution;
  2. add a fail-first repository contract proving at least one current owned production module/class set is outside coverage authority;
  3. make the smallest coverage-scope change that brings one missing module under a non-vacuous gate;
  4. add realistic tests for the real uncovered behavior revealed by the gate rather than weakening thresholds/excluding classes;
  5. repeat until the repository inventory and aggregate/per-module gates agree exactly;
  6. run the full supported OS/JDK matrix and inspect actual JaCoCo analyzed class counts, not only Maven exit status;
  7. regenerate current dependency/SBOM/SAST/security, exact-source, documentation and independent-review evidence.

Do not modify #164 merely to broaden its scope while its active writer is still establishing the non-vacuity root-cause repair. Preserve its eight-class A/B evidence as a prerequisite, then expand coverage through a safe successor or protected-main follow-up.

Primary references (APA 7)

JaCoCo. (2026). JaCoCo Maven plug-in. https://www.jacoco.org/jacoco/trunk/doc/maven.html

JaCoCo. (2026). jacoco:check. https://www.jacoco.org/jacoco/trunk/doc/check-mojo.html

JaCoCo. (2026). jacoco:report-aggregate. https://www.jacoco.org/jacoco/trunk/doc/report-aggregate-mojo.html

Relationships: #162/#164 non-vacuous ETL coverage prerequisite; #194 Mockito agent compatibility; #196 scanner completeness; #121 exact-source evidence; #159 canonical documentation/traceability; #165 release acceptance.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: accessibilityAccessibility and assistive-technology supportarea: apiAPI, protocol, event, or external contractarea: authAuthentication, authorization, identity, or tenant isolationarea: ci-cdCI, GitHub Actions, checks, release, or supply chainmaintenancepriority: mediumNormal-priority or P2 workstatus: triagedOpen issue has an organization taxonomy assignmenttype: featureNew or expanded product capability

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions