Repository navigation
✨ feat: bounded scrub-drop Matrix line source behind dashboard.matrix - #15967
Conversation
Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
Open PR overlap checkBase branch: Conflicts are checked with every PR merged onto the current tip of
|
|
hive/awaiting-ci: CI pending — sweep will merge when green. 🐝 Hive Agent: — hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88 |
There was a problem hiding this comment.
Security perspective — approve (advisory only). Read head d923601 plus pkg/github/sentinel_sweep.go, pkg/config/dashboard_matrix.go, crux/logscrub/{handler,pane}.go, and cmd/hive/main_sweeps.go. Trust boundary holds: HiveIdentified/Private are set by the sentinel (sentinel_sweep.go:102-103), not the observer; admits re-checks Active(), allowlist and both flags in Observe and again in ingest (matrix_source.go:107-111,163-165); sentinelSweepOptions is re-evaluated per sweep (main_sweeps.go:1333,1355) so flag-off reset is live. Fixture test covers every acceptance-criteria secret shape.
Two low notes, neither blocking:
-
Armour tracking only sees
+lines —matrix_source_filter.go:101-113. If a PEM-----BEGINheader is a context (unchanged) line and only the body is re-added (rebase/rotation),inArmournever flips. Body lines are 64 base64 chars and die on the entropy gate, but the final base64 line of a PEM is often <32 chars and would be emitted. Cheap fix: track armour state across all lines of the hunk (not just+), and/or also drop any line matching^[A-Za-z0-9+/]+={0,2}$of length ≥16. -
No short-secret heuristic (by design per #15934, out of scope):
matrixAcceptLine(matrix_source_filter.go:120-141) only drops on logscrub patterns, ≥32-char runs, length and markers.password: hunter2in a non-denied path (e.g.config/app.yaml) is admitted. Lines are already public on GitHub so this is amplification, not disclosure; worth a(?i)(password|passwd|secret|api[_-]?key)\s*[:=]drop rule in a follow-up.
— hive: agent=adjudicator-2 backend=copilot model=claude-fable-5.1 copilot=1.0.88
|
Hive filed review findings that do not block this PR as backlog items:
Cap: 10 backlog item(s) per PR per day. |
|
Hive status — updated in place; edits do not notify. Full history is in the Activity Monitor. Review findings from Read the two new source files against the Panic recovery leaves
Mechanism: if anything in that region panics,
Suggested fix: move the locked section into a helper that Everything else I checked looks correct to me: ring iteration order in Review findings from Conclusion: the source itself reads correct to me, but the PR fails its own CI on a structural guard and cannot merge as-is: Blocking (correctness — CI red, caused by this diff)
The repo already has the seam pattern for exactly this:
What I verified and found sound
No security, scope, or docs concerns beyond the above. Once the import is routed/allowlisted this looks ready for a maintainer. Review findings from Security perspective (head
Out of scope (pre-existing in #15933, noted for backlog): Review findings from Correctness review @ ebb006f — no blockers; one advisory. Flag-off reset has a one-observation race with an in-flight ingest ( Everything else checked out: gates match the contract ( |
…ine source Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com>
There was a problem hiding this comment.
Correctness review (head d923601)
The source logic matches #15934: gates in admits (src/pkg/dashboard/matrix_source.go:107-110), re-checked in ingest (:166), drop-only line filter (matrix_source_filter.go:120-141), ring/dedup/rate/per-head bounds all hold as written, and sentinelSweepOptions is re-evaluated per sweep (src/cmd/hive/main_sweeps.go:1333,1355) so the nil-when-off wiring is live. go build ./pkg/dashboard/ ./cmd/hive/ is clean at head.
1. CI test (rest 2/4) fails: TestDashboardImportRatchet (blocking)
src/pkg/dashboard/matrix_source_filter.go:9 adds github.com/hivecommons/hive/crux/logscrub to pkg/dashboard, which is not in src/internal/testutil/dashboard_import_allowlist.txt (crux entries at lines 10-18 stop at kubejob/paneclass). The ratchet (dashboard_import_ratchet_test.go:59) fails the shard, and coverage fails downstream ("rest=failure"). Fix: add github.com/hivecommons/hive/crux/logscrub to the allowlist between crux/kubejob and crux/paneclass (entries are sorted, line 9) and note in the PR body that #15934 mandates logscrub.ScrubString/RedactPaneTokens as the drop oracle — there is no existing seam in pkg/dashboard that exposes them.
2. Lines can land in the backlog after the flag is turned off (low, non-blocking)
ingest checks admits before taking m.mu (matrix_source.go:166-176). If MatrixObserver runs setConfig(off) + reset() (matrix_source.go:345-349) between that check and the lock, the in-flight observation is still appended and survives until the next sweep calls MatrixObserver again. Cheap fix: re-check m.cfg.Load().Active() after m.mu.Lock() and return 0 if inactive.
3. Server.Close never stops the source goroutine (low, non-blocking)
contribute_ws.go:5072-5077 closes only the contribute hub; matrixSource.close() (matrix_source.go:134) is only called from tests. Harmless for the process-lifetime singleton, but worth wiring in with the endpoint child (#15935).
— hive: agent=adjudicator backend=copilot model=claude-fable-5.1 copilot=1.0.88
ebb006f
ℹ️ Sentinel notice — informational; author is trusted (hive agent), merge is not blockedHive flagged this PR (author @hivecommons-hive[bot], head
Hive did not add the |
|
Thank you for your contribution! Your PR has been merged. We'd love to hear how your experience was: share feedback |
Why
Matrix mode (RFC #15862, @hanthor) needs a stream of real pushed lines, but those lines come from contributor patches and could carry credentials. This adds the in-memory source that turns sentinel patch observations into displayable lines, and it is built so that it can only drop content, never reveal it. The source has no HTTP surface yet; the endpoint and SSE nudge follow in #15935.
Safety invariants
sentinelSweepOptionssetsPatchObserveronly whendashboard.matrix.Active()and a dashboard server exists. Otherwise no observer is registered and nothing is created or retained. If the flag is turned off later, the existing source stops admitting observations and clears its backlog. A cmd test asserts thatPatchObserveris nil when the flag is off.SentinelPatchObservation.Files[].Patch. There are no new GitHub calls.HiveIdentified=falseare rejected inObserve, before anything is queued.!Private && cfg.Dashboard.Matrix.RepoAllowed(repo).logscrub.ScrubStringorlogscrub.RedactPaneTokenswould change a line, the line is dropped. So is any line that already has a redaction marker, contains a token-alphabet run of 32 or more characters, or is longer than 160 runes. Long lines are dropped rather than truncated, so a cut can never expose part of a token. BEGIN/END armour blocks such as PEM keys are dropped whole. Denied paths (.env*, keys/certs,id_*, secret/credential/token segments, lockfiles, vendor/node_modules, minified/generated files,exclude_paths) are skipped before the patch is parsed.Observesends to a 64-slot channel without blocking and counts drops when it is full. A single goroutine does the processing and recovers from panics. Limits: 200 lines per PR head, a 2,000-line ring, dedup sets with a 24h TTL capped at 500 PRs plus a global hash cap, 600 lines per minute, and per-observation file and patch-size caps.Tests cover the fixture-secret patch, each gate, path deny-list, dedup across re-observe and rebase, and every bound under synthetic load. They also check that a full channel never blocks, and test the wiring when the flag is off.
New dependency:
pkg/dashboardnow importscrux/logscrub(added tointernal/testutil/dashboard_import_allowlist.txt) because the drop-never-mask invariant requires the scrubber itself (ScrubString,RedactPaneTokens,ContainsRedactionMarker), and no existing seam exposes it topkg/dashboard.Closes #15934
Part of #15862 (Matrix mode RFC tracker; endpoint + SSE nudge follow in #15935)
— hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88 requested_by=@hanthor