Skip to content

test: optimize Go test performance and strengthen CI validation - #1

Open
meramsey wants to merge 6 commits into
integration/linux-build-20261020from
optimization/test-performance-20261020
Open

meramsey wants to merge 6 commits into
integration/linux-build-20261020from
optimization/test-performance-20261020

Conversation

@meramsey

@meramsey meramsey commented Oct 21, 2026 •

Copy link
Copy Markdown
Owner

Summary

Optimize the Beeper LINE test suite while preserving regression scenarios and strengthening CI validation.

Depends on beeper#271. This is a fork-local stacked PR against the actual branch backing beeper#271, meramsey/line:integration/linux-build-20261020, at 587164f7ced99bc5765ac7df4eafc11314e6d602. That branch is not available as an upstream base, so targeting upstream main now would unnecessarily repeat beeper#271's application fixes.

This diff contains only the six optimization commits ending at a7b5764a6a4c0d45d10814fd101c21eca70199b9. PR beeper#271's published branch/history is unchanged.

Performance

These are local measurements, not GitHub Actions timings. Both revisions used Go 1.26.10, CGO/libolm, GOMAXPROCS=4, bounded package parallelism -p 2, and explicit test counts that disable cached test results. Cold-cache measurements use a fresh Go build cache with module downloads already cached.

Full-suite measurement Before After Change
Normal, warm build cache 56.709 s 6.096 s 89.25% faster
Race, warm build cache 1403.551 s 146.217 s 89.58% faster
Race, cold build cache 1594.938 s 547.964 s 65.64% faster
Normal, cold build cache 384.700 s 393.692 s 2.34% slower

Warm-suite CPU consumption fell 83.08% normally and 86.38% under race instrumentation. Peak memory is approximately unchanged. Large generated-code compilation remains the cold-cache bottleneck; no substantial RAM reduction or cold-normal speedup is claimed.

Improvements

  • Establish real native heap pressure without repeatedly decoding the same cryptographic token. The original real key-allocation abort, runtime replacement, recovered signature and unchanged E2EE-memory assertions remain. Small allocator adapters call the existing native malloc/free; generated code and default heap size are unchanged.
  • Preserve the 6,000-signature reuse and 1,800-decryption stress loops. Add native allocation isolation and exhaustion/free/reuse tests.
  • Replace fixed album-preview waiting with testing/synctest quiescence, preserving concurrency and ordering assertions.
  • Isolate vector output with t.TempDir() and safely parallelize the two independent vector tests. Tests that mutate globals, environment or stdout remain sequential.
  • Add bounded fast developer feedback, complete normal/race/coverage modes, and release validation through scripts/test.sh.
  • Add complete normal/coverage and race CI jobs, package and aggregate coverage gates, mode-specific Go build caches, pinned lint-tool caches, and stale-PR cancellation.
  • Avoid duplicate feature-branch push/PR workflows; make fork Docker validation independent of private registry secrets while preserving upstream publication/deployment controls.

Validation

  • 384 passing tests/subtests, zero failures, one unchanged upstream replay-test skip in each complete normal/race run; baseline count was 382.
  • Overall statement coverage: 42.143% → 42.159%.
  • Non-generated statement coverage: 35.347% → 35.393%.
  • Zero previously covered blocks lost, including generated blocks.
  • Nine additional shuffled normal runs: 3456 passes, zero failures.
  • Two additional shuffled full race runs: 768 passes, zero failures.
  • The previously problematic eviction/context tests passed ten race repetitions each before and after optimization.
  • Full release-validation workflow passed: formatting, applicable static analysis, normal tests, race tests, coverage gates, native build and version smoke check.
  • Nine pre-commit checks passed; actionlint passed; all eight coverage-gate unit tests passed.

Test counts include parent tests and subtests. No regression scenario was removed, no package was excluded from race testing, and the existing skip was not expanded. Full test executions use uncached results; build caches are used only for compilation.

CI and repository configuration

Existing check names Lint with 1.26, Build 1.26 and build-docker remain. When adopting the workflow, add Test (normal) and Test (race) to required branch-protection checks. Deployment also depends on both modes. Repository protection settings have not been changed.

The historical upstream workflow did not run Go tests, race detection or coverage, so its duration is not an equivalent full-validation comparison. GitHub Actions performance improvements are not claimed as measured; the timing table above is from controlled local runs. Current GitHub check status should be read from this PR's checks.

Dependency and merge order

  1. Review and merge upstream fix: resolve LINE IDs and harden sends, QR login, and media beeper/line#271 first.
  2. Review these optimization-only commits against its branch here.
  3. Once fix: resolve LINE IDs and harden sends, QR login, and media beeper/line#271 is in upstream main, verify ancestry before proposing the optimization upstream. If the original parent history is retained, this branch can be used directly. If fix: resolve LINE IDs and harden sends, QR login, and media beeper/line#271 is squash-merged or rebased, create a new landing branch from updated upstream main and apply only these six optimization commits, preserving attribution and the existing published branches. GitHub cannot retarget a fork-local PR into another base repository.

Neither PR is merged automatically. The local development branch may advance for subsequent fixes, but the remote branch backing beeper#271 remains pinned to its published head.

Documentation and continued development

See docs/TEST_PERFORMANCE.md for profiling evidence, complete before/after tables, coverage invariants, stability results, cache limitations and reproducible commands.

TEST_RUN='TestResolveIdentifier' bash scripts/test.sh fast ./pkg/connector
bash scripts/test.sh normal
bash scripts/test.sh race
bash scripts/test.sh coverage
bash scripts/test.sh release

Use targeted/fast checks during implementation, full normal validation for significant changes, race validation for concurrency-sensitive changes, and the complete release workflow before publishing a release. Raw benchmark output, caches, binaries and local release artifacts are intentionally excluded from the PR.

Expose the existing native malloc/free alongside ModuleMem for bounded resource fixtures. Prefill the real signing heap, then retain the original real key-allocation abort, recovery, signature and E2EE-isolation assertions. Keep the 6000-signature and 1800-decryption stress loops unchanged. Add allocation isolation, exhaustion and coalescing/reuse tests; generated crypto and default runtime behavior are unchanged.
Use synctest quiescence to check the album-preview worker limit without scheduler-dependent sleeping. Keep ordering and concurrency assertions and bounded failure guards. Write crypto vectors to t.TempDir with mandatory write checks and test-scoped logging, enabling safe parallel execution of the two independent vector tests.
…urements

Provide fast changed-package feedback, full normal/race/coverage and release modes with uncached test execution. Enforce measured package coverage fractions and covered-statement counts for both all and non-generated code. Add seven coverage-gate tests and a serial Linux benchmark driver with isolated build caches, JSON timing, resource sampling and source-identity checks. Keep raw artifacts project-local and ignored.
Add mandatory normal/coverage and race jobs with bounded parallelism, package coverage gates and retained test artifacts. Pin/cache lint tools and static analysis, separate module and instrumentation-mode build caches, cancel stale PR runs and avoid duplicate feature-branch push runs. Keep existing lint/build check names, validate fork Docker builds without registry secrets, use GHA layer cache and gate deployment on both test modes.
Document independent cold/warm baselines, CPU and memory measurements, profiling and per-package coverage. Record 89% uncached execution improvements, unchanged cold normal bottleneck, zero lost covered blocks, repeated shuffled stability, validation commands and required CI check additions. Preserve PR beeper#271 and original release artifacts.
Apply the measured all-code and handwritten coverage floors to the aggregate as well as existing packages. Add a regression proving an uncovered new package cannot pass by leaving older package percentages unchanged. Validate both measured and release profiles against the stronger gate.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant