perf(startup): defer chunked-loader distance indexing to the lazy builder #558

Summary

Make production chunked startup honor the existing lazy distance-index lifecycle. Keep the first build on demand without regressing the race, generation and lock-order fixes already made for #149.

Verified baseline: ff1acc02255b5c6ab852e4cf931f85dbe3043ef9.

Current behavior

In cmd/server/chunked_load.go, LoadChunked acquires s.mu, restores the loaded packet/index state, then calls s.buildDistanceIndex(nodeSnapshot) synchronously before unlocking. The required path-hop/subpath readiness is marked separately after that section.

The eager distance build does not publish the lazy builder's ready/generation state. A subsequent distance request can therefore start the lazy builder again. The legacy Load path already omits this eager build, but TestDistanceIndexNotBuiltOnLoad only covers that legacy loader, not production LoadChunked.

Closed #149 repaired lazy distance races and lock ordering; it did not remove this production-loader inconsistency. Closed #120 handled transient 202 responses on the frontend, which is also a separate concern.

Bounded scope

• Remove the eager distance-index work from production chunked startup and use the established on-demand build path.
• Retain the path-hop and subpath indexes/readiness required by startup and repeater enrichment. Do not blindly remove all startup indexes or their readiness signals.
• Preserve fix(store): lazy distance build can crash (sync.Once reset during Do) and deadlock (lock order) #149's lock ordering and the current generation-aware behavior when a background load completes during a build.
• Keep distance API behavior explicit: a build in progress returns 202; once the current dataset's build/recompute is ready, the endpoint returns 200 with real distance data.
• Preserve cache invalidation and recomputer refresh ordering so the first 200 cannot serve a stale pre-load snapshot.
• No new configuration, schema changes, server DB writes or unrelated startup refactor.

Acceptance and regression tests

• Add a deterministic regression using the real LoadChunked path. Immediately after load, the distance index has not been built and no lazy build has started solely because startup completed.
• Confirm path-hop/subpath readiness remains correct and startup dependants do not acquire a new timeout or deadlock.
• Start the actual distance recomputer lifecycle, request distance data, observe 202 while the controlled build is blocked, then 200 with nonempty expected data after release.
• Concurrent first requests coalesce to one build; later requests do not perform a redundant startup build.
• Complete a background-load generation change during a controlled build and prove the old generation is not reported ready. Preserve existing fix(store): lazy distance build can crash (sync.Once reset during Do) and deadlock (lock order) #149 and generation/cache regressions.
• Run focused tests with -race, go vet and the relevant server suite. Use channels/barriers rather than sleeps to prove ordering.
• Show the new chunked-loader regression red when the eager call is restored and green without it.
• Measure isolated before/after startup distance-build counts and elapsed work on the same fixture. Report data size, environment and trade-off: startup work is deferred, not eliminated, and the first distance request pays the cold build.

Source provenance

Source repository: Kpa-clawbot/CoreScope
Issue number: 2227

Evidence boundary

The fork's current code path was inspected at the stated baseline. No new startup timing, production test or runtime benchmark was performed while preparing this issue; historical timings from other revisions must not be presented as current fork measurements.