Generator warm-build performance: incremental caching seams defeat per-edit granularity (Stage 2.5/emission CompilationProvider combines, schema-parse walks, discovery predicate, Collect barrier)

[DJGosnell]

Combined finding from the 2026-07-07 multi-agent deep review (generator compile-time performance perspective, grade C+). All items were adversarially verified (two with PARTIAL verdicts noted inline). They share the same code (QuarryGenerator pipeline wiring, ContextParser/SchemaParser, DisplayClassEnricher, UsageSiteDiscovery, PipelineOrchestrator) and the same theme: the value-equality model discipline is good, but the pipeline seams defeat incremental caching, so warm-build cost is O(total Quarry usage) rather than O(edit).

Maintainer note: Quarry is operated as a build-time generator; per-keystroke design-time regeneration is not a goal (large codebases would be too slow). Items below are framed as build-time / warm-build efficiency work. The schema-parser rework (item 2) is included for its algorithmic cost and its warm-build staleness edge under the compiler-server driver cache (VBCSCompiler GeneratorDriverCache), not as an IDE-staleness bug.

1 (high): Stage 2.5 combines with CompilationProvider — full supplemental-compilation rebuild + dataflow analysis on every run

• enrichedCallSites = rawCallSites.Collect().Combine(context.CompilationProvider).Combine(entityRegistry) (QuarryGenerator.cs:98-102). Compilation is reference-fresh every run, so DisplayClassEnricher.EnrichAll never caches. Each run re-executes:
  • BuildSupplementalCompilation (DisplayClassEnricher.cs:383-409): regenerates + re-parses the full entity/context source for every entity in every context.
  • semanticModel.AnalyzeDataFlow for every lambda and local function in every method containing a lambda-bearing Quarry call site (DisplayClassNameResolver.cs:76-118), against fresh semantic models on the brand-new supplemental compilation (zero binder reuse). Dataflow analysis is among the most expensive semantic operations Roslyn offers; the loop is single-threaded.
• At ~5,000 call sites this is plausibly seconds of serial work added to every warm build regardless of edit size — the dominant per-build generator cost.
• Fix direction (from review): move per-lambda capture classification into the per-tree discovery transform (semantic model already available there; results cache per tree); only the display-class method-ordinal computation genuinely needs cross-file data and could come from a lightweight per-type member-count provider. Alternatively cache MethodAnalysisResult keyed by (file path, method span, checksum), and cache the generated supplemental source strings keyed by EntityRegistry equality.

2 (high): Schema parsing does O(entities × project-trees) full syntax walks inside the context transform

• ContextParser.ParseContext → SchemaParser.FindAndParseSchema → FindSchemaType visits the root of every syntax tree in the compilation with a fresh walker, once per entity accessor (SchemaParser.cs:52-74, called from ContextParser.cs:137). For a project with F files and E entities, every re-parse of the context costs E×F full tree walks plus per-candidate semantic models; it also runs once per cold build.
• Because the cross-tree read happens inside a CreateSyntaxProvider transform (memoized per context tree), a warm generator driver (compiler server) can also replay stale ContextInfo/EntityInfo after schema-only edits until the context tree changes. (Accepted as low-impact for the build-time-only workflow; fixing the structure removes it for free.)
• Fix direction: discover schema classes with their own syntax provider (predicate: class deriving from Schema), emit value-equatable EntityInfo per schema tree, join contexts to schemas by name in a later Combine/Collect stage. Short-term mitigation: replace the exhaustive walk with compilation.GetSymbolsWithName(schemaName, SymbolFilter.Type) or index all Schema-derived classes once per ParseContext call.

3 (high): Discovery predicate accepts virtually every zero-argument method call

• IsQuarryMethodCandidate returns true for ANY zero-arg member-access invocation with an uppercase method name (UsageSiteDiscovery.cs:144-147) — matches x.ToString(), list.Clear(), conn.Open(), etc. Every match runs DiscoverRawCallSite, whose first step is semanticModel.GetSymbolInfo (:174); rejection happens only after symbol resolution (:307-397). Cost is per candidate node in every changed tree on every driver run, and QuarryQueryAnalyzer reuses the same predicate per edit (see analyzer issue).
• Fix direction: tighten the zero-arg branch syntactically — require the invocation to sit in a member-access chain containing at least one known Quarry method name (cheap syntax walk), or restrict receiver shapes. Target: <5% of invocations reaching GetSymbolInfo (measure hit-rate on a real corpus).

4 (high, verifier-corrected): Emission output combined with CompilationProvider defeats per-file emission caching

• RegisterImplementationSourceOutput(perFileGroups.Combine(context.CompilationProvider), ...) (QuarryGenerator.cs:144-146) makes the output pair always-new: EmitFileInterceptors re-runs for every FileInterceptorGroup every run (the :139 comment claiming per-file regeneration is inaccurate), including a linear compilation.SyntaxTrees scan per group (:519-527, O(files × groups)) and HasQuarryTrace scanning all trees per group (:950-959).
• Verifier corrections to fold into the design: (a) display-class names DO participate in group equality (via CarrierPlan → ClauseExtractionPlan → CapturedVariableExtractor.Equals), so removing the Combine would not by itself replay stale UnsafeAccessor names for rebuilt groups; (b) the things the Combine actually keeps fresh at emission are HasQuarryTrace, TraceCapture, and PipelineErrorBag side-channels; (c) the genuine latent staleness (cross-partial ordinal shifts not invalidating Stage 5) exists with or without the Combine and is tracked in the display-class issue.
• Fix direction: make enrichment data part of the equatable model (see display-class issue), pass the few compilation-derived facts (QUARRY_TRACE defined; per-file tree identity) as small equatable providers, then drop the Combine so per-file emission caching works. Coordinate with the diagnostics-pipeline issue (side-channels are currently drained here).

5 (medium): Stage 4 Collect() barrier re-analyzes all chains although chains are provably file-local

• entityRegistry.Combine(translatedCallSites.Collect()) (QuarryGenerator.cs:134-137): one changed TranslatedCallSite re-runs AnalyzeAndGroupTranslated over ALL sites — ChainAnalyzer grouping, SqlAssembler rendering every variant of every chain (up to 256/chain), CarrierAnalyzer, MySQL marker rewrite. ChainId always begins with the file path (UsageSiteDiscovery.cs:2088, 2124-2156) and ChainAnalyzer.cs:167-169 notes chains never span files — so sites can be grouped by (file, context) BEFORE a per-group transform, restoring per-file incrementality for stages 4–5d with no semantic loss.

6 (medium): Cached IR retains Roslyn syntax nodes

• RawCallSite.EnrichmentLambda is never cleared (RawCallSite.cs:250; enricher clears only EnrichmentInvocation, DisplayClassEnricher.cs:271), rooting full syntax trees (red tree → whole file) in the node tables between runs. Also (low) cached models retain Roslyn Location/Diagnostic objects in places, and per-site MD5.Create() for UniqueId generation.
• Fix: clear EnrichmentLambda after enrichment; sweep cached models for Roslyn object retention; hoist a static hasher (or use XxHash-style non-crypto hash).

7 (high, meta): No incrementality tests or warm-run benchmarks — every regression above is invisible to CI

• WithTrackingName / TrackIncrementalGeneratorSteps appear nowhere in src/. Quarry.Benchmarks.GeneratorHarness creates a fresh driver and runs once (GeneratorBenchmarkBase.cs:73-80) — cold runs only. There is no benchmark that runs the driver, applies a one-file edit (compilation.ReplaceSyntaxTree), and re-runs — the scenario that dominates warm-build cost.
• Fix: add WithTrackingName on each pipeline stage + tests asserting Cached/Unchanged step reasons after no-op and localized edits (coordinate with the test-suite issue, which covers rewriting the hollow IncrementalCachingTests); add a warm-rebuild benchmark to the GeneratorHarness.

Acceptance criteria

• Warm rebuild after a one-line edit in one query file re-runs bind/translate/analysis only for that file's sites (verified via tracked steps).
• No Combine(CompilationProvider) on Stage 2.5 or emission outputs (or a documented justification per remaining use).
• Discovery predicate hit-rate measured and reduced; GetSymbolInfo no longer invoked for arbitrary zero-arg calls.
• GeneratorHarness includes an incremental (edit + re-run) benchmark tracked on the dashboard.