Repository navigation
Decode covdata in-process for indexing speed, and fix nine review-found bugs - #1
Merged
Merged
Conversation
…cess as fallback Window assembly shelled out to 'go tool covdata textfmt' once per test, each invocation re-reading the package's whole-closure meta blob and serialising a text profile dominated by zero-count lines that ParseProfile then discarded. On a large workspace that per-test round-trip is most of the indexing wall clock. The meta and counter files are now decoded in this process: the meta is read once per package and each window costs one small counter-file read touching only the units the test actually executed. Both formats are version-stamped and unchanged since Go 1.20; every read is bounds-checked, and any structural surprise — future version, foreign magic, meta-hash mismatch, truncation — falls back to the covdata subprocess path wholesale, so an incompatible future toolchain costs time, never correctness. A parity test drives the real toolchain end to end and requires the decoder's blocks to equal the textfmt path's exactly, init window and skipped-test windows included. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01211tSDyqM7KnfziwEjPzDF
Two silent under-selection holes, both found by review: An edit inside an existing test's body was classified narrowable, but index records never contain _test.go paths — coverage instruments production code only — so attribution found nothing, the package was dropped, and the edited test never ran. Test files now take the blocking path: the owning package runs in full. Comment-only test edits still drop safely. A go.mod anywhere in the tree, matched by base name alone, forced a whole-workspace run — including fixtures under testdata that the build never reads. Testdata module files now attribute to their owning package like any other fixture. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01211tSDyqM7KnfziwEjPzDF
… module files Records intern absolute paths, but the fingerprint hashed nothing root-dependent: a second checkout sharing the cache directory got byte-identical fingerprints, hit records whose every path missed, and silently narrowed to nothing. The root is now part of every fingerprint. A go.mod under a package's testdata was classified unowned and hashed into every package's fingerprint, so editing one fixture invalidated the whole index. It now attributes to its owning package. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01211tSDyqM7KnfziwEjPzDF
Narrowing to TestMain ran zero tests and printed a green cycle: it passes the Test-prefix check but -test.run never selects it. It now declines refinement like Benchmark and Example do. A crashed run — panic, -test.timeout dump, os.Exit — credited passes to every requested test the crash prevented from running, fabricating the exact conflicting evidence the flake journal exists to detect. Passes are now credited only when the failing binary printed its FAIL trailer, the marker that the whole list executed. A staging binary left by a killed build could be promoted as fresh when a later build of a now-testless package produced no output. The staging path is cleared before every build. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01211tSDyqM7KnfziwEjPzDF
The deferred batch cleanup iterated the live variable that the harness path nils, so every default-mode run leaked one temp directory per batch — the overlays plus a compiled test binary per covering test package. The cleanup now iterates its own copy; the strong-suite test doubles as the leak check. syntaxPath indexed GoFiles as a positional fallback for Syntax, two lists that disagree in length and order for any cgo package, splicing mutations into whatever file sat at that index. The path now comes from the parsed tree's own position information, and cgo-generated cache files yield no mutants. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01211tSDyqM7KnfziwEjPzDF
Observations batched in one write share a clock reading, and the stable sort kept file order among ties — so a group trimmed at the cap kept its oldest evidence and discarded its newest, the opposite of the documented contract. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01211tSDyqM7KnfziwEjPzDF
git prints numstat paths relative to the repository toplevel no matter where it runs, while risk keys its lookups relative to the analysis root. With --root pointing anywhere below the toplevel every lookup missed, every function counted as stable, and the report printed a green verdict produced by a key mismatch rather than by coverage. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01211tSDyqM7KnfziwEjPzDF
Color was decided from stdout's TTY-ness while run, verify and the progress line write to stderr: 'assay run 2>log' wrote escapes into the log, and 'assay run >results' stripped color from a summary still heading to a terminal. Stderr-bound output now consults stderr. The --jobs help claimed a GOMAXPROCS default; the collector has defaulted to half of GOMAXPROCS since the inner build parallelism split was introduced. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01211tSDyqM7KnfziwEjPzDF
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01211tSDyqM7KnfziwEjPzDF
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Indexing performance on large workspaces
assay indexshelled out togo tool covdata textfmtonce per test: each subprocess re-read the package's whole-closure coverage meta and serialised a text profile dominated by zero-count lines thatParseProfileimmediately discarded. That cost scales with tests × dependency-closure size — the product that buries large workspaces. BENCHMARKS.md had already named this follow-up and deferred it because the binary formats are internal to the Go toolchain.This PR builds the in-process decoder for the
covmeta/covcountersbinary formats (version-stamped, unchanged since Go 1.20), with the maintenance trap fenced rather than walked into blind:Measured on a synthetic monorepo-shaped workspace (30 test packages × 20 tests, each closing over 61 instrumented packages, warm build caches, 4 cores): 13.1s → 8.5s wall, user+sys CPU 41.3s → 25.7s. At a 201-package closure: 8.7s → 5.7s. The per-test conversion tax is gone entirely; what remains is compile, link and test execution. Numbers are recorded in BENCHMARKS.md.
Bug fixes
Each fix carries a regression test (the cgo one is covered by the corpus tests); the leak test was verified red against the old code.
Silent under-selection — the failure mode this tool must never have:
_test.go, so attribution found nothing and the package dropped — the edited test never ran. Test-file edits now run the owning package in full (comment-only edits still drop safely).Wrong verdicts:
TestMainran zero tests and printed a green cycle; it now declines refinement like Benchmark/Example.-test.timeoutdump,os.Exit) credited passes to requested tests it never reached, fabricating the exact conflicting evidence the flake journal exists to detect. Passes are credited only when the failing binary printed itsFAILtrailer.git log --numstatprints toplevel-relative paths while risk keys lookups root-relative, so--root <subdir>zeroed all churn and printed a green report produced by a key mismatch. Churn is now keyed relative to the analysis root.Resource and attribution bugs:
syntaxPathfell back to indexingGoFilespositionally forSyntax, two lists that disagree for cgo packages, splicing mutations into the wrong file. The path now comes from the parsed tree's own position info; cgo-generated cache files yield no mutants..tmpbinary left by a killed build could be promoted as fresh when a later build of a now-testless package produced no output. The staging path is cleared before every build.Smaller:
go.modundertestdata(fixture, never read by the build) forced a whole-workspace run and, hashed as unowned, invalidated the entire index on any fixture edit. It now attributes to its owning package.assay run 2>logwrote escapes into the log. Stderr-bound output now consults stderr.--jobshelp claimed a GOMAXPROCS default; the real default has been half of GOMAXPROCS since the inner build-parallelism split.Testing
go vet ./...clean, fullgo test ./... -count=1green.assay indexon this repo (17 packages / 349 tests), then a warm rerun reusing all 17 records in 5ms.🤖 Generated with Claude Code
https://claude.ai/code/session_01211tSDyqM7KnfziwEjPzDF
Generated by Claude Code