perf(graphrag,mcp,storage,ui): scale topology to 100-200 services + labelled, searchable map#124
Merged
Merged
Conversation
The live OpenSSF record (https://www.bestpractices.dev/projects/12646.json) is now passing (badge_level=passing, 100%). Reconcile the repo golden source to match the authoritative live schema: - Drop 29 vestigial per-criterion <crit>_url keys. The live record has NO per-criterion URL field (only the top-level homepage_url / repo_url / badge_url / project_page_url), confirming the URL must live inline in the justification -- which it now does for all url-required criteria (PR #122). The 4 real top-level URL metadata fields are kept. - Correct _comment to document the inline-URL reality (no separate per-criterion URL field) and drop the false "<key>_url where required" clause. - Simplify report_archive link to /issues to match the live passing value. - Refresh self_assessment_date to 2026-06-18. Kept our richer project-specific justifications rather than overwriting them with bestpractices.dev's auto-generated boilerplate (and a stray OSPS tag); both validate and the badge stays passing. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LDQVJrixs2nJoea67a8pEG
…abelled, searchable map Hardens both the SQLite (single-node) and Postgres (scaled) paths for a rolling 7-day window over 100-200 services, and makes the service map usable at that scale. Backend - graphrag: add edgesByFrom/edgesByTo/opsByService adjacency indexes to ServiceStore, maintained at UpsertCallEdge/UpsertOperation/EnsureCallEdge. CallEdgesFrom/To and the new OperationsForService become O(deg) lookups, so ServiceMap drops from O(N*E+N*ops) to O(N+E+ops) and ImpactAnalysis BFS from O(N^2) to O(N+E). ServiceMap at 200 svc / 800 edges benchmarks ~53us. - storage: replace the near-useless bare span.status index with composite (tenant_id, status, start_time); unify the trace/orphan-span purge onto the batched LIMIT loop for ALL drivers so SQLite no longer holds the single writer lock for the whole retention sweep and stalls ingest. - storage: add Postgres BRIN indexes on spans(start_time)/traces(timestamp) for the by-age retention DELETE; raise pool MaxIdleConns 10->25 and add ConnMaxIdleTime (DB_CONN_MAX_IDLE_TIME, default 10m). - config: default IngestPipelinePerTenantCap to ~30% of the resolved queue when unset (explicit =0 still disables) so one noisy tenant cannot fill the queue. MCP - emit compact JSON (Marshal vs MarshalIndent) across all tool responses. - honour the documented get_service_map "service" focus arg via depth-bounded ServiceMapAround; no focus -> full map (backward compatible). - cache trace_graph (immutable completed-trace topology; errors not cached). UI - service map dots now render a haloed, truncated, a11y label whose visibility is zoom-scaled by phyllotaxis criticality band (most-critical label first; emphasized/search-matched always label). - add a deferred client-side service search that highlights matches + neighbours and dims the rest, reusing the existing emphasis machinery, with match count, empty state, and clear. Tests: graphrag/mcp/storage/config/ingest suites green; UI builds + 339 UI tests pass; changed files lint-clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EGHhZg96LhYRPQoG8x4nyP
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
undici 7.25.0 — a transitive dev dependency via the Vite/test toolchain in ui/package-lock.json — is flagged by two newly-published advisories: GHSA-vmh5-mc38-953g (High, 7.4) and GHSA-pr7r-676h-xcf6 (Medium, 5.9), both fixed in 7.28.0. In-range bump (the parent already allows ^7.25.0), lockfile only. npm audit reports 0 vulnerabilities; tsc + vitest (339 tests) green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LDQVJrixs2nJoea67a8pEG
The composite-index comment added mid-struct in this PR broke gofmt tag alignment on the Span block (golangci-lint gofmt failure). Pure formatting; no behaviour change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LDQVJrixs2nJoea67a8pEG
rfEdges computed the active path only from selection/impact, so a search accented matched NODES while their connections stayed at resting style — the 'searchable map' was half-done at 100-200 services where edges are the signal. Fold searchMatches into the edge active test (precedence: impact > search > selection, mirroring the node emphasis ladder) and into the memo deps. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LDQVJrixs2nJoea67a8pEG
…ection) CodeQL flagged factory.go:170 logging the operator-set DB_DRIVER string (rule go/log-injection, medium) — the merge-blocking unresolved code-scanning thread. Strip CR/LF via a local var before logging so a malformed value can't forge log lines; the counts/duration are non-injectable. Operator config (not a remote-input path), but fix-over-suppress per the security policy, and the local-var sanitizer is a barrier CodeQL recognises, clearing the alert at the source. Drops the now-redundant #nosec G706. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LDQVJrixs2nJoea67a8pEG
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Hardens both the SQLite (single-node) and PostgreSQL (scaled) paths for a rolling 7-day window over 100–200 services, and makes the service map usable at that scale.
Backend
edgesByFrom/edgesByTo/opsByServiceadjacency indexes toServiceStore, maintained atUpsertCallEdge/UpsertOperation/EnsureCallEdge.CallEdgesFrom/Toand the newOperationsForServicebecome O(deg) lookups, soServiceMapdrops from O(N·E+N·ops) → O(N+E+ops) andImpactAnalysisBFS from O(N²) → O(N+E).ServiceMapat 200 svc / 800 edges benchmarks ~53µs.span.statusindex with composite(tenant_id, status, start_time); unify the trace/orphan-span purge onto the batchedLIMITloop for all drivers so SQLite no longer holds the single writer lock for the whole retention sweep (which stalled ingest).spans(start_time)/traces(timestamp)for the by-age retention DELETE; raise poolMaxIdleConns10→25 and addConnMaxIdleTime(DB_CONN_MAX_IDLE_TIME, default 10m).IngestPipelinePerTenantCapto ~30% of the resolved queue when unset (explicit=0still disables) so one noisy tenant can't fill the queue.MCP
MarshalvsMarshalIndent) across all tool responses.get_service_mapservicefocus arg via depth-boundedServiceMapAround; no focus → full map (backward compatible).trace_graph(immutable completed-trace topology; errors not cached).UI
Testing
graphrag/mcp/storage/config/ingestGo suites green (incl. newadjacency_index_test.goparity test +BenchmarkServiceMap).Notes
(tenant_id, trace_id, span_id)constraint makes it unsafe — same reason partitioning was restricted tologs).rebuildRowLimitconfig knob,NOT EXISTS+traces(trace_id)index swap, threading the MCP per-callctxinto GORM.🤖 Generated with Claude Code
https://claude.ai/code/session_01EGHhZg96LhYRPQoG8x4nyP