Skip to content

[QA] Complete deferred coverage, fuzzing, conformance, and runtime validation #400

Description

@ty-everett

Purpose

This issue is the focused successor to the final QA phase deferred from #324. Deferral did not waive any gate. Resume this program only after the non-QA modernization and contributor-policy closeout are on main.

Preserved WIP branches origin/codex/improve-test-coverage and origin/codex/stack-wide-fuzzing are evidence to audit and selectively rebase, not pre-approved changes. Preserve public API, wire, serialization, persistence, errors, defaults, runtime targets, browser/mobile behavior, public-service access, and compatibility with other implementations.

Work program

  • Re-measure aggregate line/branch/function coverage and risk-tier coverage from current main; inventory every critical 0%-covered production module.
  • Reach and enforce overall >=80% coverage and Tier 0/1 >=85% line / >=80% branch coverage, or replace a numeric target only with a stricter reviewed risk-based contract that proves equivalent protection.
  • Complete high-value coverage for cryptography, Script/consensus, transaction and wire codecs, wallet signing/storage/persistence, remote input, authentication/payment, overlay synchronization, WASM/worker, browser/mobile, and service trust boundaries.
  • Expand meaningful fuzz/property testing across every suitable parser, codec, serializer, framing, validation, graph/sync, and destination boundary. Require reproducible seeds/corpora, minimized regressions, bounded PR budgets, longer scheduled campaigns, and mutation evidence that assertions detect semantic defects.
  • Evaluate and add true coverage-guided fuzzing where it adds behavior discovery beyond the governed property suites; document targets where property/model/differential testing is the stronger method.
  • Audit every manual/live/resource suite and Wallet Toolbox operator suite. Keep useful operator code outside test discovery, remove only genuinely obsolete material, and give every retained suite an owner, environment contract, safe command, expected assertions, schedule, and artifact retention.
  • Resolve or explicitly disposition all intended conformance skips with implementation-independent evidence; require 100% pass for required vectors and exact generated counts.
  • Prove there are no assertion-free or incomplete required tests, anonymous/expired skips, empty expected suites, accidental network tests, or unowned manual/live cases.
  • Complete deep CLI behavior, browser/bundler/mobile/WASM worker runtime validation and execute stable benchmark campaigns on suitable hosted hardware with variance policy and machine-readable history.
  • Review test sharding and scheduled campaign cost after correctness is complete; optimize without dropping assertions, coverage, environments, or analysis.
  • Update generated facts, package documentation, test-quality governance, replay instructions, and operator guidance with the exact final state.

Required evidence

  • Exact main SHA and merged PR inventory.
  • Full required-check run with all applicable contexts terminal green, zero new Sonar findings/hotspots, and zero new CodeQL alerts.
  • Aggregate and per-risk-tier coverage report, critical-module inventory, and patch-gate evidence.
  • Fuzz/property target and corpus inventory, campaign duration/iterations, seeds, crashes/regressions, mutation results, and scheduled workflow evidence.
  • Manual/live/operator suite disposition ledger and execution evidence for every retained critical journey.
  • Conformance report with required pass rate and owned intended-skip counts matching generated documentation.
  • Browser/mobile/CLI/WASM runtime and benchmark reports with stable environments and variance policy.
  • Compatibility self-review confirming no unsupported public behavior change.

Exit

Close only when all final QA gates above are implemented and evidenced on merged main. Do not close from a local run, a WIP branch, an aggregate quality badge, or a refreshed baseline.

Metadata

Metadata

Assignees

Labels

hardeningReliability, governance, and release hardening

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions