fix(ui+graphrag): sunflower service map at scale — flow direction, blast-radius highlight, topology survival at 1% sampling#117
Merged
Conversation
Cross-service CALLS edges in the service map were only observed when BOTH
the caller and callee spans survived the per-service sampler. The sampler
runs at the top of TraceServer.Export() and `continue`s past dropped spans
before the span model is built or handed to spanCallback/GraphRAG, so at low
sample rates the joint survival of a parent+child pair is vanishingly small
and the in-memory topology is starved — service nodes render with almost no
edges (no visible flow direction). The 60s DB rebuild cannot recover it
because the DB also only ever received sampled spans.
Add a bounded, pre-sample topology observer in the graphrag package. The OTLP
trace receiver now calls GraphRAG.ObserveSpanTopology for EVERY received span
BEFORE the sampler's keep/drop decision, on both the async pipeline path and
the synchronous fallback. The observer maintains a per-tenant bounded LRU
(map[spanID]serviceName, cap 100k, container/list LRU mirroring drain.go) and,
for each span whose parent is a known span in a different service, guarantees
the parentService->childService CALLS edge EXISTS — deduped per (src->tgt)
pair so work and memory are bounded by the number of distinct service-pairs.
Edge existence is guaranteed via new existence-only ServiceStore methods
EnsureService and EnsureCallEdge, which create the node/edge with zeroed
aggregates if absent and are no-ops otherwise. Edge aggregates
(CallCount/latency/error-rate) remain exclusively owned by the sampled path
(processSpan -> UpsertCallEdge) and the 60s DB rebuild, so the observer never
pollutes them. Error/slow always-on sampling, DB persistence (sampled), and
per-tenant store isolation are unchanged.
The observer is wired into TraceServer via a function-typed hook
(SetTopologyObserver) to keep internal/ingest decoupled from internal/graphrag,
matching the existing spanCallback injection pattern; main.go wires it to
graphRAG.ObserveSpanTopology.
Tests (TDD):
- TestObserveSpanTopology_EdgeSurvivesSampling: checkout->payments edge present
when no sampled span ever formed it.
- TestObserveSpanTopology_PreservesSampledAggregates: observing a pair with an
existing sampled edge does not inflate its aggregates.
- TestObserveSpanTopology_BoundedLRUEvicts: per-tenant spanID->service LRU
evicts oldest past cap.
- TestObserveSpanTopology_DedupBoundsEdges: repeated pair observation keeps one
edge.
- TestObserveSpanTopology_TenantIsolation: edges/parent lookups do not leak
across tenants.
- TestObserveSpanTopology_{UnknownParentNoEdge,SameServiceNoEdge}: negative
cases.
- ingest TestTopologyObserver_FiresPreSample_{Async,Sync}Path: the hook fires
for both spans before a ~99%-drop sampler on both ingest paths.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LDQVJrixs2nJoea67a8pEG
… selection
Replace the dagre layered layout with the deterministic phyllotaxis
("sunflower") layout. At 120+ services a layered DAG collapsed sparse/
disconnected graphs into a single ~8000px-tall column (all nodes land in
dagre rank 0 under rankdir:LR), which fitView clamped to minZoom — so the
dense view was an unreadable vertical line of status dots. The sunflower
fills a disc evenly from 7 to 120+ services (most-critical at the centre),
so the map stays a navigable disc at scale.
- Wire the existing, tested radialLayout (layoutRadial) into FlowMap; edges
become straight chords pinned to node borders at chip zoom and centre-to-
centre at dot zoom (no routing waypoints).
- Direction now rides a static ARROWHEAD (markerEnd) on every edge, coloured
to match its stroke — legible at any zoom and under prefers-reduced-motion.
Previously the marching-dash animation was the only cue, and it vanishes
when zoomed out and for reduced-motion users.
- Selection + blast radius now emphasise EDGES too: the active path (1-hop of
the selection, or both endpoints in the impact cone) is accent-coloured,
thickened, opaque and animated; off-path edges dim to a faint hairline.
Neighbour and in-cone NODES gain an accent ring (previously unstyled);
selected/root rings are stronger.
- Remove the now-dead dagre engine (dagreLayout.ts) and the @dagrejs/dagre
dependency; the ConstellationHome lazy chunk drops ~13 KB gz (72->60),
budget cap tightened 76->66.
Pairs with the backend topology-observer fix (b54f04e) so cross-service edges
exist to draw even at low sampling. Verified: tsc, eslint, 339 vitest tests,
budgets, jscpd 0.42%, npm audit 0 — FlowMap regression suite unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LDQVJrixs2nJoea67a8pEG
|
aksOps
added a commit
that referenced
this pull request
Jun 18, 2026
…gy observer (#118) Update the project instructions to match the code after #117: - The service map uses the deterministic phyllotaxis ("sunflower") layout (radialLayout.ts), not the (now-removed) dagre layered layout; edges are border-pinned chords with direction arrowheads; selection accents the active path (nodes + edges). @dagrejs/dagre is removed. - Document the pre-sample topology observer (topology_observer.go) that records cross-service CALLS edges independent of SAMPLING_RATE, why it exists (joint-survival starvation at low sampling), and its existence-only + bounded design so sampled aggregates and OOM bounds are unaffected. Claude-Session: https://claude.ai/code/session_01LDQVJrixs2nJoea67a8pEG Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
At 120+ services the service map collapsed to a single ~8000px-tall vertical line of dots, flow direction was invisible, and selecting a node barely highlighted anything. Root causes (confirmed by a read-only analysis pass):
rankdir:LRthat's one column. Real topologies are sparse, so ~all 120 nodes stacked; fitView then clamped to minZoom (< the 0.62 LOD threshold) → all dots.prefers-reduced-motion.Fix
Backend —
b54f04e(fix(graphrag): observe service topology independent of sampling): a bounded, pre-sample topology observer records each service→service edge from the first span per pair, independent of the sampler. Existence-only (EnsureService/EnsureCallEdge, zeroed stats) so it never pollutes the sampled aggregates; bounded by a per-tenant spanID→service LRU (cap 100k, samecontainer/listpattern as the Drain miner) + per-pair dedup, so memory ≈ number of service-pairs. Does not feed TraceStore/eventCh, so the SQLite OOM-survival design is untouched. Tenant-isolated.Frontend —
c39630c(feat(ui): sunflower service-map layout + directional edges + stronger selection):markerEnd) on every edge, coloured to match stroke — legible at any zoom and under reduced motion.@dagrejs/dagredep → ConstellationHome chunk −13 KB gz (72→60).Validation (NO REGRESSION)
SAMPLING_RATE=0.01: all 8 cross-service edges present (backend fix), arrowheads show direction, flaky paths render red/critical; clickinggatewaylights up its neighbourhood + edges and dims the rest.go build/vet, 678 tests (28 pkgs) +-raceon graphrag/ingest, gofmt, golangci-lint clean. New regression test drivesExport()at ~1% and asserts the cross-service edge appears.🤖 Generated with Claude Code