From eab7411958afb52a28b497ed4fd27ee754ce19a3 Mon Sep 17 00:00:00 2001 From: Andrew Anderson Date: Wed, 18 Nov 2026 14:03:29 -0500 Subject: [PATCH 1/3] =?UTF-8?q?=E2=9C=A8=20authz:=20shadow-mode=20decision?= =?UTF-8?q?=20seam=20for=20the=20push=20broker's=20changed-files=20guard?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add src/crux/authz: a flat typed Request, a Decision, a pure in-process authz.Decide evaluator that mirrors one existing decision (empty push, then protected paths), and a Shadow recorder that compares the evaluation with the existing gate, keeps one versioned attributes-only event per call on its own bounded in-memory stream, and logs divergences. Default off (HIVE_AUTHZ_SHADOW, stream size HIVE_AUTHZ_SHADOW_BUFFER). It enforces nothing and changes no behaviour: the broker calls it after its own guard has decided and never reads a result back. Fail-closed semantics are documented and tested in Decide but not applied. Parity tests replay the push broker's existing cases through the seam. Docs: design record phase status, security model, env vars, crux README, changelog fragment. Refs #18988 Signed-off-by: Andrew Anderson --- .../added-18988-authz-shadow-decide.md | 1 + src/crux/README.md | 1 + src/crux/authz/authz.go | 191 ++++++++++ src/crux/authz/decide.go | 114 ++++++ src/crux/authz/decide_test.go | 143 +++++++ src/crux/authz/event.go | 154 ++++++++ src/crux/authz/parity_test.go | 150 ++++++++ src/crux/authz/shadow.go | 254 +++++++++++++ src/crux/authz/shadow_test.go | 355 ++++++++++++++++++ src/crux/pushbroker/protected.go | 27 +- src/crux/pushbroker/pushbroker.go | 6 + src/crux/pushbroker/pushbroker_shadow_test.go | 140 +++++++ src/docs/design/README.md | 2 +- .../design/authorization-decision-point.md | 48 ++- src/docs/documentation-map.md | 2 +- src/docs/env-vars.md | 2 + src/docs/security-model.md | 23 ++ 17 files changed, 1606 insertions(+), 7 deletions(-) create mode 100644 changelog.d/added-18988-authz-shadow-decide.md create mode 100644 src/crux/authz/authz.go create mode 100644 src/crux/authz/decide.go create mode 100644 src/crux/authz/decide_test.go create mode 100644 src/crux/authz/event.go create mode 100644 src/crux/authz/parity_test.go create mode 100644 src/crux/authz/shadow.go create mode 100644 src/crux/authz/shadow_test.go create mode 100644 src/crux/pushbroker/pushbroker_shadow_test.go diff --git a/changelog.d/added-18988-authz-shadow-decide.md b/changelog.d/added-18988-authz-shadow-decide.md new file mode 100644 index 0000000000..c6c879a7b0 --- /dev/null +++ b/changelog.d/added-18988-authz-shadow-decide.md @@ -0,0 +1 @@ +- Authorization decision seam, phase 0/1 (shadow mode, off by default, [#18988](https://github.com/hivecommons/hive/issues/18988)): a new `authz` package in the Crux module adds an in-process `authz.Decide(Request) Decision` evaluator that mirrors one existing decision, the push broker's changed-files guard (empty push, then protected paths). With `HIVE_AUTHZ_SHADOW=true` (stream size `HIVE_AUTHZ_SHADOW_BUFFER`, default 1000), each time the guard decides the evaluator is run too, one versioned decision event (attributes only, never payloads, paths or credentials) is kept on its own bounded in-memory stream, and a difference from the guard is logged as a divergence. It enforces nothing and changes no behaviour, on or off; with the variable unset nothing is recorded. Parity tests replay the push broker's existing cases through the seam. The design record's phase status and the security model are updated. diff --git a/src/crux/README.md b/src/crux/README.md index 1d93579916..e481c3ca24 100644 --- a/src/crux/README.md +++ b/src/crux/README.md @@ -24,6 +24,7 @@ Forge-neutral durable event/work lanes are described in | `crux/cli` | the `crux` command: `run`, `ps`, `logs`, `kick`, `stop`, `serve`, `agy-turn` (the pane-side agy subprocess), and `--server` remote mode | | `crux/api`, `crux/serve` | the wire contract of `crux serve` (stdlib-only client and types, `api.RuntimeRoutes`: every `crux.Runtime` method → route) and the server | | `crux/api/client` | `client.Runtime`: `crux.Runtime` over the API; what `crux/runtime.New` returns for `CRUX_MODE=sidecar` | +| `crux/authz` | the authorization decision seam, shadow mode only (RFC #18988 phase 0/1): `authz.Decide` re-derives the push broker's changed-files decision, a `Shadow` compares it with the existing gate and keeps one decision event per call on a bounded stream; default off (`HIVE_AUTHZ_SHADOW`), enforces nothing, a leaf that imports no other crux package but `logscrub` | | `crux/lanes` | opt-in JetStream durable lanes: neutral project ACLs, event replay, claimable delivery groups and fenced receipts; Hive is an ordinary participant | | `crux/api/callback`, `crux/hivecap` | the hive callback contract (capabilities hive serves the runtime: scoped tokens, agent mint, Bob credential, PRs, Linear, scanners, conditions, host policy, inference routes, effects, usage) and the runtime-side adapters `crux serve --hive-callback` installs; `api.HostSetters` is the table that accounts for every `crux.Host` setter (route / callback / config key / in-process-only) | | `crux/runtime` | `runtime.New` (mode selection: `CRUX_MODE=inprocess` default, `sidecar`) and `runtime.Supervisor`, the child supervisor hive uses for `crux serve` when `CRUX_SUPERVISE=child` | diff --git a/src/crux/authz/authz.go b/src/crux/authz/authz.go new file mode 100644 index 0000000000..ee4f1732a6 --- /dev/null +++ b/src/crux/authz/authz.go @@ -0,0 +1,191 @@ +// Package authz is the seam for the authorization decision point described in +// src/docs/design/authorization-decision-point.md (RFC #18988). +// +// This is phase 0/1 of that design and it is SHADOW MODE ONLY: +// +// - Decide is a pure, in-process evaluator. It mirrors one existing decision, +// the push broker's changed-files guard (empty push, then protected paths). +// - A Shadow compares that evaluation with what the existing gate decided, +// records one versioned decision event per call on its own bounded stream, +// and logs divergences. It is OFF by default and gated by HIVE_AUTHZ_SHADOW. +// - Nothing here enforces anything. No caller reads a Decision to allow or +// refuse an action; the existing gate remains the only authority, so +// enabling or disabling the shadow changes no behaviour. +// +// Fail-closed semantics (documented, not enforcing). When a later phase lets +// the seam decide, an evaluator error or panic is a deny with reason +// authz_error, and a request that lacks a required attribute is a deny with +// reason missing_attribute. Decide already returns exactly those decisions so +// the semantics are testable now, but in this phase a shadow deny that the +// existing gate did not issue is only a logged divergence. +// +// The package is a leaf on purpose. The proxy and the push broker live in the +// same module and must be able to import it, so it imports nothing from them. +// Parity with the existing gates is proven from an external test package. +// +// Events carry attributes only: names, counts and codes. Never request bodies, +// diffs, file contents, credentials or full path lists. +package authz + +// SchemaVersion is the version of the decision event schema (Event.V). New +// fields are additive; a change of meaning bumps it. +const SchemaVersion = 1 + +// FloorVersion identifies the built-in rule set that Decide evaluates. It is +// part of every decision's PolicyVersion, so an event states which rules +// produced it. Bump it whenever a built-in rule changes. +const FloorVersion = "floor-1" + +// Effect is the outcome of a decision. +type Effect string + +const ( + // EffectAllow means the request may proceed. + EffectAllow Effect = "allow" + // EffectDeny means the request must not proceed. + EffectDeny Effect = "deny" + // EffectHold means a human must decide. It is part of the schema from the + // start so later phases add no new effect, but no built-in rule returns it + // in this phase. + EffectHold Effect = "hold" +) + +// Surface is the enforcement point a request arrives at. +type Surface string + +const ( + // SurfaceProxy is the GitHub MITM proxy. + SurfaceProxy Surface = "proxy" + // SurfaceRelay is a hive-side request relay. + SurfaceRelay Surface = "relay" + // SurfaceBroker is the sandbox push broker. + SurfaceBroker Surface = "broker" +) + +// KindBranchPush is the action kind for pushing a branch. It is the same +// spelling as effects.KindBranchPush; a test in this package pins the two +// together so there is one vocabulary, not two. +const KindBranchPush = "branch_push" + +// ForgeGitHub is the only forge the broker pushes to today. +const ForgeGitHub = "github" + +// Reason codes. Codes that exist in the proxy keep their spelling (see +// src/crux/proxy/denial_message.go); the ones below are new and namespaced +// to the broker or the seam. +const ( + // ReasonOK is the reason on an allow. + ReasonOK = "ok" + // ReasonEmptyPush is a push with no committed changes. + ReasonEmptyPush = "empty_push" + // ReasonProtectedPath is a push that touches a protected path. + ReasonProtectedPath = "protected_path" + // ReasonMissingAttribute is a request that lacks a required attribute. + ReasonMissingAttribute = "missing_attribute" + // ReasonUnsupportedAction is a request no built-in rule covers. + ReasonUnsupportedAction = "unsupported_action" + // ReasonAuthzError is an evaluator error or panic. + ReasonAuthzError = "authz_error" +) + +// Stable rule ids of the built-in floor. A decision names the rule that +// produced it. +const ( + // RuleBrokerEmptyPush is the broker's empty-push refusal. + RuleBrokerEmptyPush = "floor.broker.changed_files.empty_push" + // RuleBrokerProtectedPath is the broker's protected-path refusal. + RuleBrokerProtectedPath = "floor.broker.changed_files.protected_path" + // RuleBrokerChangedFilesPass is the broker's changed-files guard passing. + RuleBrokerChangedFilesPass = "floor.broker.changed_files.pass" + // RuleMissingAttribute is the required-attribute check. + RuleMissingAttribute = "floor.request.missing_attribute" + // RuleUnsupportedAction is the answer for an action with no built-in rule. + RuleUnsupportedAction = "floor.request.unsupported_action" + // RuleEvaluatorError is the answer when the evaluator itself failed. + RuleEvaluatorError = "floor.evaluator.error" +) + +// Request is the flat, typed input of a decision. Flat on purpose: it is the +// stable surface later operator rules are written against. It never carries +// request bodies, diffs, file contents or credentials, only attributes derived +// from them. +type Request struct { + Actor Actor + Action Action + Resource Resource + Context Context +} + +// Actor is who is acting. +type Actor struct { + // Agent is the agent name, empty when the caller cannot name one. + Agent string + // Mode is the agent permission mode when the surface knows it. + Mode string + // ACMMLevel is the ACMM level when the surface knows it, else zero. + ACMMLevel int + // Identified is true when the agent was attributed to a known name. + Identified bool +} + +// Action is what is being attempted. +type Action struct { + // Kind is the action vocabulary entry, for example KindBranchPush. + Kind string + // Surface is the enforcement point. + Surface Surface + // Forge names the forge, for example ForgeGitHub. + Forge string +} + +// Resource is what the action targets. +type Resource struct { + // Repo is owner/name. + Repo string + // Branch is the target branch. + Branch string + // Paths are the changed files for a push. Only a count of them is ever + // recorded in an event. + Paths []string +} + +// Context is the surrounding policy input the caller already holds. +type Context struct { + // RequestID correlates the event with the caller's own log line. + RequestID string + // ProtectedPaths is the effective protected path list the caller applies. + // A guard ending in "/" protects a directory tree; any other guard + // protects exactly that file. + ProtectedPaths []string +} + +// Decision is the output of Decide. +type Decision struct { + // Effect is allow, deny or hold. + Effect Effect + // RuleID is the stable id of the rule that produced the decision. + RuleID string + // ReasonCode is the machine-readable reason. + ReasonCode string + // PolicyVersion identifies the rule set and policy inputs used. + PolicyVersion string + // MatchedPaths is how many changed paths a path rule matched. + MatchedPaths int + // Shadow is true when the decision was computed but not enforced. A Shadow + // observer sets it; in this phase every recorded decision is shadow. + Shadow bool +} + +// Verdict is what the existing gate decided, expressed in the seam's terms so +// the two can be compared. The existing gates have no rule ids or policy +// version, so the comparison is on effect and reason code only. +type Verdict struct { + Effect Effect + ReasonCode string +} + +// Matches reports whether the evaluation agrees with the existing gate: the +// same effect and the same reason code. +func (v Verdict) Matches(d Decision) bool { + return v.Effect == d.Effect && v.ReasonCode == d.ReasonCode +} diff --git a/src/crux/authz/decide.go b/src/crux/authz/decide.go new file mode 100644 index 0000000000..74c908a42f --- /dev/null +++ b/src/crux/authz/decide.go @@ -0,0 +1,114 @@ +package authz + +import ( + "crypto/sha256" + "encoding/hex" + "path" + "path/filepath" + "slices" + "strings" +) + +// policyVersionHexLen is how many hex characters of the policy-input digest are +// kept in PolicyVersion. Twelve is enough to tell two protected-path lists +// apart in an event and short enough to read. +const policyVersionHexLen = 12 + +// Decide evaluates one request against the built-in floor and returns the +// decision. It is pure: no I/O, no logging, no shared state, safe to call from +// any goroutine. It never panics; an evaluator panic becomes a deny with +// reason authz_error (the documented fail-closed posture). +// +// In this phase only the push broker's changed-files guard has a rule. Any +// other action returns a deny with reason unsupported_action, which a Shadow +// never compares against an existing gate (a caller only observes decisions +// it has a legacy answer for). +func Decide(req Request) Decision { + return decideWith(req, evaluateFloor) +} + +// decideWith runs eval and converts a panic into the fail-closed decision. +// The evaluator is a parameter so a test can prove the recovery without +// wiring a panic into the real floor. +func decideWith(req Request, eval func(Request) Decision) (d Decision) { + defer func() { + if recover() != nil { + d = Decision{ + Effect: EffectDeny, + RuleID: RuleEvaluatorError, + ReasonCode: ReasonAuthzError, + PolicyVersion: policyVersion(req), + } + } + }() + return eval(req) +} + +func evaluateFloor(req Request) Decision { + version := policyVersion(req) + if strings.TrimSpace(req.Resource.Repo) == "" || strings.TrimSpace(req.Resource.Branch) == "" { + return Decision{Effect: EffectDeny, RuleID: RuleMissingAttribute, ReasonCode: ReasonMissingAttribute, PolicyVersion: version} + } + if req.Action.Surface == SurfaceBroker && req.Action.Kind == KindBranchPush { + return brokerChangedFiles(req, version) + } + return Decision{Effect: EffectDeny, RuleID: RuleUnsupportedAction, ReasonCode: ReasonUnsupportedAction, PolicyVersion: version} +} + +// brokerChangedFiles mirrors the push broker's changed-files guard in +// src/crux/pushbroker/protected.go: an empty push is refused first, then a +// push that touches a protected path. The matching below is written +// independently of pushbroker.ProtectedPathViolations on purpose, so that a +// parity test (and the live shadow) can catch the two drifting apart. +func brokerChangedFiles(req Request, version string) Decision { + if len(req.Resource.Paths) == 0 { + return Decision{Effect: EffectDeny, RuleID: RuleBrokerEmptyPush, ReasonCode: ReasonEmptyPush, PolicyVersion: version} + } + matched := 0 + for _, file := range req.Resource.Paths { + if touchesProtected(file, req.Context.ProtectedPaths) { + matched++ + } + } + if matched > 0 { + return Decision{Effect: EffectDeny, RuleID: RuleBrokerProtectedPath, ReasonCode: ReasonProtectedPath, PolicyVersion: version, MatchedPaths: matched} + } + return Decision{Effect: EffectAllow, RuleID: RuleBrokerChangedFilesPass, ReasonCode: ReasonOK, PolicyVersion: version} +} + +// touchesProtected reports whether file is covered by any guard. Both sides +// are cleaned the way the broker cleans them. A guard written with a trailing +// slash covers the whole directory tree under it; any other guard covers +// exactly that one path, so ".github/workflows" without the slash does not +// cover ".github/workflows/ci.yml". +func touchesProtected(file string, guards []string) bool { + clean := normalise(file) + for _, guard := range guards { + g := normalise(guard) + if strings.HasSuffix(guard, "/") || strings.HasSuffix(g, "/") { + if strings.HasPrefix(clean, strings.TrimSuffix(g, "/")+"/") { + return true + } + continue + } + if clean == g { + return true + } + } + return false +} + +func normalise(p string) string { + return strings.TrimPrefix(path.Clean(filepath.ToSlash(p)), "./") +} + +// policyVersion names the rule set and the policy inputs a decision used: +// the floor version plus a short digest of the protected path list. The list +// is sorted and de-duplicated first, so order does not change the version. +func policyVersion(req Request) string { + guards := slices.Clone(req.Context.ProtectedPaths) + slices.Sort(guards) + guards = slices.Compact(guards) + sum := sha256.Sum256([]byte(strings.Join(guards, "\x00"))) + return FloorVersion + ":" + hex.EncodeToString(sum[:])[:policyVersionHexLen] +} diff --git a/src/crux/authz/decide_test.go b/src/crux/authz/decide_test.go new file mode 100644 index 0000000000..0a5c99ffc6 --- /dev/null +++ b/src/crux/authz/decide_test.go @@ -0,0 +1,143 @@ +package authz + +import ( + "strings" + "testing" +) + +var testGuards = []string{".github/workflows/", "OWNERS", "policies/"} + +func brokerRequest(paths []string, guards []string) Request { + return Request{ + Actor: Actor{Agent: "scanner", Identified: true}, + Action: Action{Kind: KindBranchPush, Surface: SurfaceBroker, Forge: ForgeGitHub}, + Resource: Resource{Repo: "org/name", Branch: "work", Paths: paths}, + Context: Context{ProtectedPaths: guards}, + } +} + +func TestDecideBrokerChangedFiles(t *testing.T) { + cases := []struct { + name string + paths []string + guards []string + effect Effect + rule string + reason string + matched int + }{ + {"empty push is refused first", nil, testGuards, EffectDeny, RuleBrokerEmptyPush, ReasonEmptyPush, 0}, + {"empty push wins over a protected guard", []string{}, testGuards, EffectDeny, RuleBrokerEmptyPush, ReasonEmptyPush, 0}, + {"directory guard covers the tree", []string{".github/workflows/ci.yml"}, testGuards, EffectDeny, RuleBrokerProtectedPath, ReasonProtectedPath, 1}, + {"file guard covers exactly that file", []string{"OWNERS"}, testGuards, EffectDeny, RuleBrokerProtectedPath, ReasonProtectedPath, 1}, + {"file guard does not cover a nested path", []string{"OWNERS/sub.txt"}, testGuards, EffectAllow, RuleBrokerChangedFilesPass, ReasonOK, 0}, + {"guard without slash does not cover a tree", []string{".github/workflows/ci.yml"}, []string{".github/workflows"}, EffectAllow, RuleBrokerChangedFilesPass, ReasonOK, 0}, + {"dot-slash prefix is cleaned", []string{"./policies/a.rego"}, testGuards, EffectDeny, RuleBrokerProtectedPath, ReasonProtectedPath, 1}, + {"dot-dot segments are cleaned", []string{"docs/../policies/a.rego"}, testGuards, EffectDeny, RuleBrokerProtectedPath, ReasonProtectedPath, 1}, + {"matching is case sensitive", []string{"owners"}, testGuards, EffectAllow, RuleBrokerChangedFilesPass, ReasonOK, 0}, + {"a lookalike directory is allowed", []string{"policies-old/a.rego"}, testGuards, EffectAllow, RuleBrokerChangedFilesPass, ReasonOK, 0}, + {"unprotected push passes", []string{"pkg/safe.go", "README.md"}, testGuards, EffectAllow, RuleBrokerChangedFilesPass, ReasonOK, 0}, + {"every protected file is counted", []string{"OWNERS", "pkg/safe.go", "policies/x"}, testGuards, EffectDeny, RuleBrokerProtectedPath, ReasonProtectedPath, 2}, + {"no guards protects nothing", []string{"OWNERS"}, nil, EffectAllow, RuleBrokerChangedFilesPass, ReasonOK, 0}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + got := Decide(brokerRequest(tc.paths, tc.guards)) + if got.Effect != tc.effect || got.RuleID != tc.rule || got.ReasonCode != tc.reason || got.MatchedPaths != tc.matched { + t.Fatalf("Decide = %+v, want effect=%s rule=%s reason=%s matched=%d", got, tc.effect, tc.rule, tc.reason, tc.matched) + } + if got.Shadow { + t.Fatal("Decide must not mark a decision shadow; only a Shadow observer does") + } + if !strings.HasPrefix(got.PolicyVersion, FloorVersion+":") { + t.Fatalf("PolicyVersion = %q, want the floor version prefix", got.PolicyVersion) + } + }) + } +} + +func TestDecideFailsClosedOnMissingAttribute(t *testing.T) { + for name, mutate := range map[string]func(*Request){ + "no repo": func(r *Request) { r.Resource.Repo = "" }, + "blank repo": func(r *Request) { r.Resource.Repo = " " }, + "no branch": func(r *Request) { r.Resource.Branch = "" }, + "blank branch": func(r *Request) { r.Resource.Branch = "\t" }, + "no repo, benign": func(r *Request) { r.Resource.Repo = ""; r.Resource.Paths = []string{"a.go"} }, + } { + t.Run(name, func(t *testing.T) { + req := brokerRequest([]string{"a.go"}, testGuards) + mutate(&req) + got := Decide(req) + if got.Effect != EffectDeny || got.ReasonCode != ReasonMissingAttribute || got.RuleID != RuleMissingAttribute { + t.Fatalf("Decide = %+v, want deny missing_attribute", got) + } + }) + } +} + +func TestDecideUnsupportedActionDenies(t *testing.T) { + for name, mutate := range map[string]func(*Request){ + "other kind": func(r *Request) { r.Action.Kind = "pull_request_create" }, + "other surface": func(r *Request) { r.Action.Surface = SurfaceProxy }, + "zero action": func(r *Request) { r.Action = Action{} }, + } { + t.Run(name, func(t *testing.T) { + req := brokerRequest([]string{"a.go"}, testGuards) + mutate(&req) + got := Decide(req) + if got.Effect != EffectDeny || got.ReasonCode != ReasonUnsupportedAction || got.RuleID != RuleUnsupportedAction { + t.Fatalf("Decide = %+v, want deny unsupported_action", got) + } + }) + } +} + +func TestDecideWithRecoversPanicAsDeny(t *testing.T) { + req := brokerRequest([]string{"a.go"}, testGuards) + got := decideWith(req, func(Request) Decision { panic("evaluator bug") }) + if got.Effect != EffectDeny || got.ReasonCode != ReasonAuthzError || got.RuleID != RuleEvaluatorError { + t.Fatalf("decideWith = %+v, want deny authz_error", got) + } + if got.PolicyVersion == "" { + t.Fatal("a fail-closed decision still names its policy version") + } +} + +func TestDecideNeverReturnsHoldOrAllowOnError(t *testing.T) { + for _, req := range []Request{ + {}, + brokerRequest(nil, nil), + brokerRequest([]string{"a"}, nil), + } { + if got := Decide(req); got.Effect == EffectHold { + t.Fatalf("no built-in rule returns hold in this phase: %+v", got) + } + } + if got := Decide(Request{}); got.Effect != EffectDeny { + t.Fatalf("the zero request must deny, got %+v", got) + } +} + +func TestPolicyVersionIsOrderIndependentAndInputSensitive(t *testing.T) { + a := policyVersion(brokerRequest(nil, []string{"x/", "y", "y"})) + b := policyVersion(brokerRequest(nil, []string{"y", "x/"})) + if a != b { + t.Fatalf("order or duplicates changed the version: %q vs %q", a, b) + } + if c := policyVersion(brokerRequest(nil, []string{"x/"})); c == a { + t.Fatalf("a different list must change the version: %q", c) + } +} + +func TestVerdictMatches(t *testing.T) { + d := Decision{Effect: EffectDeny, ReasonCode: ReasonProtectedPath} + if !(Verdict{Effect: EffectDeny, ReasonCode: ReasonProtectedPath}).Matches(d) { + t.Fatal("same effect and reason must match") + } + if (Verdict{Effect: EffectDeny, ReasonCode: ReasonEmptyPush}).Matches(d) { + t.Fatal("a different reason must not match") + } + if (Verdict{Effect: EffectAllow, ReasonCode: ReasonProtectedPath}).Matches(d) { + t.Fatal("a different effect must not match") + } +} diff --git a/src/crux/authz/event.go b/src/crux/authz/event.go new file mode 100644 index 0000000000..648e69e338 --- /dev/null +++ b/src/crux/authz/event.go @@ -0,0 +1,154 @@ +package authz + +import ( + "crypto/rand" + "encoding/hex" + "fmt" + "strings" + "sync/atomic" + "time" + "unicode" + "unicode/utf8" + + "github.com/hivecommons/hive/crux/logscrub" +) + +const ( + // maxAttrBytes bounds every free-text attribute copied into an event. + maxAttrBytes = 256 + // attrPreScrubBytes bounds the text handed to the credential scrubber, so a + // hostile oversized value cannot make recording expensive. + attrPreScrubBytes = 1024 + // decisionIDRandomBytes is the entropy in a decision id. + decisionIDRandomBytes = 8 + // decisionIDPrefix marks a decision id. + decisionIDPrefix = "dec_" + // ParityMatch and ParityMismatch are the Event.Parity values. + ParityMatch = "match" + ParityMismatch = "mismatch" +) + +// Event is the versioned decision record, one per Observe call. It holds +// attributes only. There is deliberately no field that could carry a request +// body, a diff, file contents, a credential or a path list: the changed files +// appear only as a count. +type Event struct { + V int `json:"v"` + TS time.Time `json:"ts"` + DecisionID string `json:"decision_id"` + Surface string `json:"surface"` + Effect string `json:"effect"` + Shadow bool `json:"shadow"` + RuleID string `json:"rule_id"` + ReasonCode string `json:"reason_code"` + PolicyVersion string `json:"policy_version"` + Actor EventActor `json:"actor"` + Action EventAction `json:"action"` + Resource EventResource `json:"resource"` + // LatencyUS is how long Decide took, in microseconds. + LatencyUS int64 `json:"latency_us"` + // Legacy is what the existing gate decided. + Legacy EventLegacy `json:"legacy"` + // Parity is ParityMatch or ParityMismatch. + Parity string `json:"parity"` + RequestID string `json:"request_id,omitempty"` +} + +// EventActor is the actor attributes of an event. +type EventActor struct { + Agent string `json:"agent,omitempty"` + Mode string `json:"mode,omitempty"` + ACMMLevel int `json:"acmm_level,omitempty"` + Identified bool `json:"identified"` +} + +// EventAction is the action attributes of an event. +type EventAction struct { + Kind string `json:"kind"` + Forge string `json:"forge,omitempty"` +} + +// EventResource is the resource attributes of an event. PathsCount is how many +// changed paths the request carried and MatchedCount how many a path rule +// matched; the paths themselves are never recorded. +type EventResource struct { + Repo string `json:"repo"` + Branch string `json:"branch"` + PathsCount int `json:"paths_count"` + MatchedCount int `json:"matched_count,omitempty"` +} + +// EventLegacy is the existing gate's verdict. +type EventLegacy struct { + Effect string `json:"effect"` + ReasonCode string `json:"reason_code"` +} + +var decisionSeq atomic.Uint64 + +// newDecisionID returns a unique id for a decision. If the system random +// source fails it falls back to a process-local sequence, which is still +// unique within the process. +func newDecisionID() string { + buf := make([]byte, decisionIDRandomBytes) + if _, err := rand.Read(buf); err != nil { + return fmt.Sprintf("%sseq%d", decisionIDPrefix, decisionSeq.Add(1)) + } + return decisionIDPrefix + hex.EncodeToString(buf) +} + +// attr makes free text safe to record: control characters dropped, credential +// shapes masked, length bounded. It is applied to every string that originates +// outside the evaluator (agent, repo, branch, request id). +func attr(s string) string { + s = clip(s, attrPreScrubBytes) + s = strings.Map(func(r rune) rune { + if unicode.IsControl(r) { + return -1 + } + return r + }, s) + return clip(logscrub.ScrubString(s), maxAttrBytes) +} + +// clip truncates s to at most limit bytes without splitting a rune. +func clip(s string, limit int) string { + if len(s) <= limit { + return s + } + cut := limit + for cut > 0 && !utf8.RuneStart(s[cut]) { + cut-- + } + return s[:cut] +} + +func buildEvent(ts time.Time, req Request, d Decision, legacy Verdict, latency time.Duration) Event { + parity := ParityMismatch + if legacy.Matches(d) { + parity = ParityMatch + } + return Event{ + V: SchemaVersion, + TS: ts.UTC(), + DecisionID: newDecisionID(), + Surface: attr(string(req.Action.Surface)), + Effect: string(d.Effect), + Shadow: d.Shadow, + RuleID: d.RuleID, + ReasonCode: d.ReasonCode, + PolicyVersion: d.PolicyVersion, + Actor: EventActor{ + Agent: attr(req.Actor.Agent), + Mode: attr(req.Actor.Mode), + ACMMLevel: req.Actor.ACMMLevel, + Identified: req.Actor.Identified, + }, + Action: EventAction{Kind: attr(req.Action.Kind), Forge: attr(req.Action.Forge)}, + Resource: EventResource{Repo: attr(req.Resource.Repo), Branch: attr(req.Resource.Branch), PathsCount: len(req.Resource.Paths), MatchedCount: d.MatchedPaths}, + LatencyUS: latency.Microseconds(), + Legacy: EventLegacy{Effect: string(legacy.Effect), ReasonCode: legacy.ReasonCode}, + Parity: parity, + RequestID: attr(req.Context.RequestID), + } +} diff --git a/src/crux/authz/parity_test.go b/src/crux/authz/parity_test.go new file mode 100644 index 0000000000..a49a34578d --- /dev/null +++ b/src/crux/authz/parity_test.go @@ -0,0 +1,150 @@ +package authz_test + +import ( + "fmt" + "slices" + "strings" + "testing" + + "github.com/hivecommons/hive/crux/authz" + "github.com/hivecommons/hive/crux/effects" + "github.com/hivecommons/hive/crux/pushbroker" +) + +// This file is the parity corpus for the first mirrored decision, the push +// broker's changed-files guard (RFC #18988 phase 0/1). It lives in an external +// test package so it can import pushbroker, which itself imports authz. + +// existingChangedFilesVerdict is the push broker's guard as it stands today +// (pushbroker.Broker.guardChangedFiles): an empty push is refused first, then +// a push that touches a protected path, using the exported matcher the broker +// itself calls. The second return is the list the broker would record as +// ProtectedReject. +func existingChangedFilesVerdict(files, guards []string) (authz.Verdict, []string) { + if len(files) == 0 { + return authz.Verdict{Effect: authz.EffectDeny, ReasonCode: authz.ReasonEmptyPush}, nil + } + if rejected := pushbroker.ProtectedPathViolations(files, guards); len(rejected) > 0 { + return authz.Verdict{Effect: authz.EffectDeny, ReasonCode: authz.ReasonProtectedPath}, rejected + } + return authz.Verdict{Effect: authz.EffectAllow, ReasonCode: authz.ReasonOK}, nil +} + +func brokerPushRequest(files, guards []string) authz.Request { + return authz.Request{ + Actor: authz.Actor{Agent: "scanner", Identified: true}, + Action: authz.Action{Kind: authz.KindBranchPush, Surface: authz.SurfaceBroker, Forge: authz.ForgeGitHub}, + Resource: authz.Resource{Repo: "org/name", Branch: "work", Paths: files}, + Context: authz.Context{ProtectedPaths: guards}, + } +} + +// assertParity fails when the seam and the existing guard disagree on effect, +// reason code, or how many files were rejected. +func assertParity(t *testing.T, label string, files, guards []string) { + t.Helper() + want, rejected := existingChangedFilesVerdict(files, guards) + got := authz.Decide(brokerPushRequest(files, guards)) + if !want.Matches(got) { + t.Errorf("%s: files=%q guards=%q: seam = %s/%s, existing guard = %s/%s", label, files, guards, got.Effect, got.ReasonCode, want.Effect, want.ReasonCode) + } + if got.MatchedPaths != len(rejected) { + t.Errorf("%s: files=%q guards=%q: seam matched %d paths, existing guard rejected %d (%q)", label, files, guards, got.MatchedPaths, len(rejected), rejected) + } +} + +func TestKindBranchPushIsTheEffectsKind(t *testing.T) { + if authz.KindBranchPush != effects.KindBranchPush { + t.Fatalf("authz.KindBranchPush = %q, effects.KindBranchPush = %q: one action vocabulary, not two", authz.KindBranchPush, effects.KindBranchPush) + } +} + +// TestParityPushBrokerChangedFiles replays the cases the broker's own tests +// already encode (pushbroker_test.go and pushbroker_coverage_test.go) through +// the seam. Every row must produce the same (effect, reason code) as the guard. +func TestParityPushBrokerChangedFiles(t *testing.T) { + defaults := pushbroker.DefaultProtectedPaths + corpus := []struct { + name string + files []string + guards []string + }{ + {"TestProtectedPathViolationsTable", []string{"policies/guard.yaml", "OWNERS", "hive.yaml.dashboard", "pkg/safe.go"}, defaults}, + {"TestBrokerRejectsProtectedPaths", []string{".github/workflows/ci.yaml"}, defaults}, + {"quoted names", []string{".github/workflows/dépl.yml", `policies/a"b.md`, "policies/c\td.md"}, defaults}, + {"heuristic fixture is not a path rule", []string{"fixture_test.go"}, defaults}, + {"custom protected override", []string{"docs/notes.md"}, []string{"docs/"}}, + {"custom override leaves defaults unprotected", []string{"OWNERS"}, []string{"docs/"}}, + {"empty push", nil, defaults}, + {"empty slice", []string{}, defaults}, + } + for _, protected := range []string{".github/workflows/ci.yml", "policies/x.rego", "OWNERS", "bin/gh-wrapper.sh"} { + // TestBrokerRejectsRenamedProtectedPaths: both rename endpoints are checked. + corpus = append(corpus, struct { + name string + files []string + guards []string + }{"rename endpoints " + protected, []string{protected, "notes/moved.txt"}, defaults}) + } + for _, tc := range corpus { + t.Run(tc.name, func(t *testing.T) { assertParity(t, tc.name, tc.files, tc.guards) }) + } +} + +// TestParityPushBrokerPathShapes crosses protected-path guards with the ways a +// path can be spelled, so a difference in cleaning or in the directory-versus- +// file rule shows up as a mismatch rather than going unnoticed. +func TestParityPushBrokerPathShapes(t *testing.T) { + guardSets := [][]string{ + pushbroker.DefaultProtectedPaths, + {"docs/"}, + {".github/workflows"}, + {"./policies/"}, + {"a/b/"}, + {"OWNERS", "OWNERS/"}, + {"/"}, + {"."}, + {"./"}, + {""}, + {"../"}, + nil, + } + var files []string + for _, guard := range pushbroker.DefaultProtectedPaths { + bare := strings.TrimSuffix(guard, "/") + files = append(files, + guard, bare, bare+"/child", bare+"/deep/er/child.txt", "./"+bare, "./"+bare+"/child", + "x/"+bare, "x/"+bare+"/child", strings.ToUpper(bare), bare+"-old", bare+"x/child", bare+"//child", + "sub/../"+bare, "sub/../"+bare+"/child") + } + files = append(files, "", ".", "..", "../x", "/", "/abs/policies/x", "a/../OWNERS", "a//b", "docs/readme.md", "a/b/c", "a/b") + + checked := 0 + for gi, guards := range guardSets { + for _, file := range files { + assertParity(t, fmt.Sprintf("guardset %d single", gi), []string{file}, guards) + checked++ + } + assertParity(t, fmt.Sprintf("guardset %d all", gi), files, guards) + assertParity(t, fmt.Sprintf("guardset %d safe only", gi), []string{"pkg/safe.go", "README.md"}, guards) + assertParity(t, fmt.Sprintf("guardset %d empty", gi), nil, guards) + } + if checked == 0 { + t.Fatal("the shape corpus is empty") + } +} + +// TestParityHoldsForPolicyVersionAcrossListOrder guards the one input the +// existing guard does not have: the version. Reordering the protected list must +// change nothing about the effect or the reason. +func TestParityHoldsForPolicyVersionAcrossListOrder(t *testing.T) { + forward := slices.Clone(pushbroker.DefaultProtectedPaths) + reversed := slices.Clone(forward) + slices.Reverse(reversed) + files := []string{"OWNERS", "pkg/a.go"} + a := authz.Decide(brokerPushRequest(files, forward)) + b := authz.Decide(brokerPushRequest(files, reversed)) + if a != b { + t.Fatalf("list order changed the decision: %+v vs %+v", a, b) + } +} diff --git a/src/crux/authz/shadow.go b/src/crux/authz/shadow.go new file mode 100644 index 0000000000..df8e2c3683 --- /dev/null +++ b/src/crux/authz/shadow.go @@ -0,0 +1,254 @@ +package authz + +import ( + "log/slog" + "os" + "strconv" + "strings" + "sync" + "sync/atomic" + "time" +) + +const ( + // EnvShadow turns shadow mode on when set to a true value. Unset, empty or + // anything unparseable is off. Read once per process by DefaultShadow. + EnvShadow = "HIVE_AUTHZ_SHADOW" + // EnvShadowBuffer overrides the number of events the stream keeps. + EnvShadowBuffer = "HIVE_AUTHZ_SHADOW_BUFFER" + + // DefaultBufferEvents is the stream capacity when EnvShadowBuffer is unset + // or invalid. + DefaultBufferEvents = 1000 + // MaxBufferEvents caps EnvShadowBuffer so the stream stays small. + MaxBufferEvents = 10000 + // maxLoggedDivergenceKinds bounds how many distinct divergence shapes are + // logged. Every divergence is still counted; only the log line is limited, + // so a hot mismatch cannot flood the log. + maxLoggedDivergenceKinds = 256 +) + +// Sink receives each event after it is appended to the stream. It is optional +// and is the module boundary a later phase uses to hand events to the hive. An +// implementation must not block; a panic in it is recovered and counted and +// never reaches the caller. +type Sink interface { + Record(Event) +} + +// Stats is a point-in-time copy of a Shadow's counters. +type Stats struct { + // Observed is the number of decisions recorded. + Observed uint64 + // Mismatches is the number of decisions that differed from the existing gate. + Mismatches uint64 + // Dropped is the number of events evicted from the bounded stream. + Dropped uint64 + // Errors is the number of internal recording failures that were contained. + Errors uint64 + // Buffered is how many events the stream holds now. + Buffered int + // Capacity is the stream's bound. + Capacity int +} + +// Shadow is the shadow-mode recorder. A nil *Shadow is the OFF state: every +// method is a safe no-op, nothing is recorded, nothing is allocated. Callers +// hold a possibly nil *Shadow and call Observe unconditionally. +type Shadow struct { + // Logger receives divergence logs. Nil uses slog.Default(). + Logger *slog.Logger + // Sink optionally receives every event. + Sink Sink + // Now supplies event timestamps. Nil uses time.Now. + Now func() time.Time + + mu sync.Mutex + ring []Event + head int + count int + capacity int + logged map[string]struct{} + + observed atomic.Uint64 + mismatches atomic.Uint64 + dropped atomic.Uint64 + errors atomic.Uint64 +} + +// NewShadow returns an enabled Shadow whose stream keeps at most capacity +// events. A non-positive capacity uses DefaultBufferEvents and anything above +// MaxBufferEvents is clamped. +func NewShadow(capacity int) *Shadow { + if capacity <= 0 { + capacity = DefaultBufferEvents + } + capacity = min(capacity, MaxBufferEvents) + return &Shadow{ + ring: make([]Event, capacity), + capacity: capacity, + logged: make(map[string]struct{}), + } +} + +// ShadowFromEnv builds a Shadow from the environment, or returns nil (off) +// unless EnvShadow parses as true. Config-gated and default off. +func ShadowFromEnv(getenv func(string) string) *Shadow { + if getenv == nil { + return nil + } + on, err := strconv.ParseBool(strings.TrimSpace(getenv(EnvShadow))) + if err != nil || !on { + return nil + } + capacity := DefaultBufferEvents + if raw := strings.TrimSpace(getenv(EnvShadowBuffer)); raw != "" { + if n, err := strconv.Atoi(raw); err == nil && n > 0 { + capacity = n + } + } + return NewShadow(capacity) +} + +var processShadow = sync.OnceValue(func() *Shadow { return ShadowFromEnv(os.Getenv) }) + +// DefaultShadow is the process-wide shadow built from the environment on first +// use. It is nil (off) unless EnvShadow is true, and a change to the +// environment after the first call has no effect until the process restarts. +func DefaultShadow() *Shadow { + return processShadow() +} + +// Observe evaluates req with Decide, compares the result with the verdict the +// existing gate already reached, records one event on the stream and logs a +// divergence. It enforces nothing and returns nothing: the existing gate's +// answer is the only one any caller acts on. +// +// It is safe on a nil receiver (off), safe for concurrent use, and contains +// its own failures: a panic anywhere in recording is recovered and counted, +// never propagated, so enabling the shadow cannot change what the caller does. +func (s *Shadow) Observe(req Request, legacy Verdict) { + if s == nil { + return + } + defer func() { + if recover() != nil { + s.errors.Add(1) + } + }() + start := time.Now() + d := Decide(req) + d.Shadow = true + latency := time.Since(start) + + ev := buildEvent(s.now(), req, d, legacy, latency) + s.observed.Add(1) + s.append(ev) + if ev.Parity == ParityMismatch { + s.mismatches.Add(1) + s.logDivergence(ev) + } + s.deliver(ev) +} + +// Events returns a copy of the stream, oldest first. Nil when off. +func (s *Shadow) Events() []Event { + if s == nil { + return nil + } + s.mu.Lock() + defer s.mu.Unlock() + if s.capacity == 0 { + return nil + } + out := make([]Event, 0, s.count) + start := (s.head - s.count + s.capacity) % s.capacity + for i := range s.count { + out = append(out, s.ring[(start+i)%s.capacity]) + } + return out +} + +// Stats returns the counters. The zero Stats when off. +func (s *Shadow) Stats() Stats { + if s == nil { + return Stats{} + } + s.mu.Lock() + buffered := s.count + s.mu.Unlock() + return Stats{ + Observed: s.observed.Load(), + Mismatches: s.mismatches.Load(), + Dropped: s.dropped.Load(), + Errors: s.errors.Load(), + Buffered: buffered, + Capacity: s.capacity, + } +} + +func (s *Shadow) append(ev Event) { + s.mu.Lock() + defer s.mu.Unlock() + if s.capacity == 0 { + return + } + s.ring[s.head] = ev + s.head = (s.head + 1) % s.capacity + if s.count < s.capacity { + s.count++ + return + } + s.dropped.Add(1) +} + +func (s *Shadow) deliver(ev Event) { + if s.Sink == nil { + return + } + defer func() { + if recover() != nil { + s.errors.Add(1) + } + }() + s.Sink.Record(ev) +} + +func (s *Shadow) logDivergence(ev Event) { + key := strings.Join([]string{ev.RuleID, ev.Effect, ev.Legacy.Effect, ev.Legacy.ReasonCode}, "|") + s.mu.Lock() + _, seen := s.logged[key] + if !seen && len(s.logged) < maxLoggedDivergenceKinds { + s.logged[key] = struct{}{} + } else { + seen = true + } + s.mu.Unlock() + if seen { + return + } + logger := s.Logger + if logger == nil { + logger = slog.Default() + } + logger.Warn("authz shadow: decision differs from the existing gate; existing gate stays in force", + "decision_id", ev.DecisionID, + "surface", ev.Surface, + "kind", ev.Action.Kind, + "rule_id", ev.RuleID, + "shadow_effect", ev.Effect, + "shadow_reason", ev.ReasonCode, + "legacy_effect", ev.Legacy.Effect, + "legacy_reason", ev.Legacy.ReasonCode, + "repo", ev.Resource.Repo, + "branch", ev.Resource.Branch, + "agent", ev.Actor.Agent, + "policy_version", ev.PolicyVersion) +} + +func (s *Shadow) now() time.Time { + if s.Now != nil { + return s.Now() + } + return time.Now() +} diff --git a/src/crux/authz/shadow_test.go b/src/crux/authz/shadow_test.go new file mode 100644 index 0000000000..899b4123cd --- /dev/null +++ b/src/crux/authz/shadow_test.go @@ -0,0 +1,355 @@ +package authz + +import ( + "bytes" + "encoding/json" + "fmt" + "log/slog" + "strings" + "sync" + "testing" + "time" + "unicode/utf8" +) + +var ( + legacyAllow = Verdict{Effect: EffectAllow, ReasonCode: ReasonOK} + legacyProtected = Verdict{Effect: EffectDeny, ReasonCode: ReasonProtectedPath} + fixedShadowTime = time.Date(2026, 11, 18, 14, 31, 2, 0, time.UTC) + fixedShadowClock = func() time.Time { return fixedShadowTime } + secretShapedToken = "ghp_" + strings.Repeat("A1b2", 9) +) + +func newTestShadow(capacity int) (*Shadow, *bytes.Buffer) { + var logs bytes.Buffer + s := NewShadow(capacity) + s.Logger = slog.New(slog.NewTextHandler(&logs, nil)) + s.Now = fixedShadowClock + return s, &logs +} + +func TestShadowOffRecordsNothing(t *testing.T) { + var off *Shadow + off.Observe(brokerRequest([]string{"OWNERS"}, testGuards), legacyProtected) + if got := off.Events(); got != nil { + t.Fatalf("a nil Shadow recorded %d events", len(got)) + } + if got := off.Stats(); got != (Stats{}) { + t.Fatalf("a nil Shadow has counters: %+v", got) + } +} + +func TestShadowFromEnvIsOffByDefault(t *testing.T) { + for name, env := range map[string]map[string]string{ + "unset": {}, + "empty": {EnvShadow: ""}, + "false": {EnvShadow: "false"}, + "garbage": {EnvShadow: "yes please"}, + "buffer alone": {EnvShadowBuffer: "50"}, + } { + t.Run(name, func(t *testing.T) { + if s := ShadowFromEnv(func(k string) string { return env[k] }); s != nil { + t.Fatalf("ShadowFromEnv = %+v, want nil (off)", s) + } + }) + } + if s := ShadowFromEnv(nil); s != nil { + t.Fatal("a nil getenv must be off") + } +} + +func TestShadowFromEnvOnAndBufferBounds(t *testing.T) { + cases := []struct { + buffer string + want int + }{ + {"", DefaultBufferEvents}, + {"25", 25}, + {"0", DefaultBufferEvents}, + {"-4", DefaultBufferEvents}, + {"lots", DefaultBufferEvents}, + {fmt.Sprint(MaxBufferEvents * 10), MaxBufferEvents}, + } + for _, tc := range cases { + s := ShadowFromEnv(func(k string) string { + switch k { + case EnvShadow: + return "true" + case EnvShadowBuffer: + return tc.buffer + } + return "" + }) + if s == nil { + t.Fatalf("buffer %q: shadow is off with %s=true", tc.buffer, EnvShadow) + } + if got := s.Stats().Capacity; got != tc.want { + t.Fatalf("buffer %q: capacity = %d, want %d", tc.buffer, got, tc.want) + } + } +} + +func TestShadowRecordsOneVersionedEventPerCall(t *testing.T) { + s, logs := newTestShadow(10) + req := brokerRequest([]string{"pkg/a.go", "OWNERS"}, testGuards) + req.Context.RequestID = "req-1" + s.Observe(req, legacyProtected) + + events := s.Events() + if len(events) != 1 { + t.Fatalf("events = %d, want exactly one per call", len(events)) + } + ev := events[0] + if ev.V != SchemaVersion || !ev.TS.Equal(fixedShadowTime) { + t.Fatalf("version/ts = %d/%v", ev.V, ev.TS) + } + if !strings.HasPrefix(ev.DecisionID, decisionIDPrefix) { + t.Fatalf("decision id = %q", ev.DecisionID) + } + if ev.Surface != string(SurfaceBroker) || ev.Effect != string(EffectDeny) || !ev.Shadow || + ev.RuleID != RuleBrokerProtectedPath || ev.ReasonCode != ReasonProtectedPath { + t.Fatalf("event = %+v", ev) + } + if ev.Resource.PathsCount != 2 || ev.Resource.MatchedCount != 1 || ev.Resource.Repo != "org/name" || ev.Resource.Branch != "work" { + t.Fatalf("resource = %+v", ev.Resource) + } + if ev.Actor.Agent != "scanner" || !ev.Actor.Identified || ev.Action.Kind != KindBranchPush || ev.RequestID != "req-1" { + t.Fatalf("actor/action = %+v %+v", ev.Actor, ev.Action) + } + if ev.Parity != ParityMatch || ev.Legacy.Effect != string(EffectDeny) || ev.Legacy.ReasonCode != ReasonProtectedPath { + t.Fatalf("parity/legacy = %q %+v", ev.Parity, ev.Legacy) + } + if ev.LatencyUS < 0 { + t.Fatalf("latency = %d", ev.LatencyUS) + } + if st := s.Stats(); st.Observed != 1 || st.Mismatches != 0 || st.Dropped != 0 || st.Errors != 0 { + t.Fatalf("stats = %+v", st) + } + if logs.Len() != 0 { + t.Fatalf("a matching decision must not log a divergence: %s", logs.String()) + } +} + +func TestShadowEventJSONKeysAreStable(t *testing.T) { + s, _ := newTestShadow(2) + s.Observe(brokerRequest([]string{"pkg/a.go"}, testGuards), legacyAllow) + raw, err := json.Marshal(s.Events()[0]) + if err != nil { + t.Fatal(err) + } + var got map[string]any + if err := json.Unmarshal(raw, &got); err != nil { + t.Fatal(err) + } + for _, key := range []string{"v", "ts", "decision_id", "surface", "effect", "shadow", "rule_id", "reason_code", "policy_version", "actor", "action", "resource", "latency_us", "legacy", "parity"} { + if _, ok := got[key]; !ok { + t.Fatalf("event JSON is missing %q: %s", key, raw) + } + } + if v, ok := got["v"].(float64); !ok || int(v) != SchemaVersion { + t.Fatalf("v = %v", got["v"]) + } +} + +func TestShadowEventCarriesAttributesOnly(t *testing.T) { + s, _ := newTestShadow(2) + req := brokerRequest([]string{"secret-looking-dir/very-private-file-name.txt", "OWNERS"}, testGuards) + req.Actor.Agent = "agent-" + secretShapedToken + req.Resource.Branch = "topic/" + secretShapedToken + "\nforged log line" + req.Resource.Repo = "org/name" + req.Context.RequestID = "id-" + secretShapedToken + s.Observe(req, legacyProtected) + + raw, err := json.Marshal(s.Events()[0]) + if err != nil { + t.Fatal(err) + } + text := string(raw) + if strings.Contains(text, secretShapedToken) { + t.Fatalf("a credential-shaped value reached the event: %s", text) + } + if strings.Contains(text, "very-private-file-name") || strings.Contains(text, "secret-looking-dir") { + t.Fatalf("a changed path reached the event, only a count may: %s", text) + } + if ev := s.Events()[0]; strings.ContainsAny(ev.Resource.Branch, "\n\r") || strings.ContainsAny(ev.Actor.Agent+ev.RequestID, "\n\r") { + t.Fatalf("a control character survived into the event: %q", ev.Resource.Branch) + } +} + +func TestShadowAttributesAreBounded(t *testing.T) { + s, _ := newTestShadow(2) + req := brokerRequest([]string{"a.go"}, testGuards) + req.Resource.Branch = strings.Repeat("b", maxAttrBytes*8) + // Three-byte runes, so the byte limit falls inside a rune and truncation + // has to step back to a boundary. + req.Actor.Agent = strings.Repeat("\u20ac", maxAttrBytes) + s.Observe(req, legacyAllow) + ev := s.Events()[0] + if len(ev.Resource.Branch) > maxAttrBytes || len(ev.Actor.Agent) > maxAttrBytes { + t.Fatalf("attributes not bounded: branch=%d agent=%d", len(ev.Resource.Branch), len(ev.Actor.Agent)) + } + if !utf8.ValidString(ev.Actor.Agent) || !strings.HasPrefix(ev.Actor.Agent, "\u20ac") { + t.Fatalf("a rune was split by truncation: %q", ev.Actor.Agent) + } +} + +func TestShadowLogsToTheDefaultLoggerWhenNoneIsSet(t *testing.T) { + var logs bytes.Buffer + previous := slog.Default() + slog.SetDefault(slog.New(slog.NewTextHandler(&logs, nil))) + t.Cleanup(func() { slog.SetDefault(previous) }) + + s := NewShadow(2) + s.Observe(brokerRequest([]string{"OWNERS"}, testGuards), legacyAllow) + if !strings.Contains(logs.String(), "authz shadow") { + t.Fatalf("divergence not logged to the default logger: %q", logs.String()) + } +} + +func TestShadowStreamIsBounded(t *testing.T) { + const capacity, total = 3, 10 + s, _ := newTestShadow(capacity) + for i := range total { + req := brokerRequest([]string{"a.go"}, testGuards) + req.Resource.Branch = fmt.Sprintf("b%d", i) + s.Observe(req, legacyAllow) + } + events := s.Events() + if len(events) != capacity { + t.Fatalf("stream holds %d events, want the bound %d", len(events), capacity) + } + for i, ev := range events { + if want := fmt.Sprintf("b%d", total-capacity+i); ev.Resource.Branch != want { + t.Fatalf("event %d branch = %q, want %q (oldest first, newest kept)", i, ev.Resource.Branch, want) + } + } + st := s.Stats() + if st.Observed != total || st.Dropped != total-capacity || st.Buffered != capacity || st.Capacity != capacity { + t.Fatalf("stats = %+v", st) + } +} + +func TestShadowLogsDivergenceAndEnforcesNothing(t *testing.T) { + s, logs := newTestShadow(10) + // The evaluator denies a protected path; pretend the existing gate allowed. + req := brokerRequest([]string{"OWNERS"}, testGuards) + s.Observe(req, legacyAllow) + + ev := s.Events()[0] + if ev.Parity != ParityMismatch || ev.Effect != string(EffectDeny) || ev.Legacy.Effect != string(EffectAllow) { + t.Fatalf("event = %+v", ev) + } + if st := s.Stats(); st.Mismatches != 1 { + t.Fatalf("mismatches = %d, want 1", st.Mismatches) + } + out := logs.String() + for _, want := range []string{"authz shadow", "existing gate stays in force", "shadow_effect=deny", "legacy_effect=allow", "rule_id=" + RuleBrokerProtectedPath} { + if !strings.Contains(out, want) { + t.Fatalf("divergence log missing %q: %s", want, out) + } + } +} + +func TestShadowDivergenceLogIsDeduplicatedButStillCounted(t *testing.T) { + s, logs := newTestShadow(10) + for range 5 { + s.Observe(brokerRequest([]string{"OWNERS"}, testGuards), legacyAllow) + } + if got := strings.Count(logs.String(), "authz shadow"); got != 1 { + t.Fatalf("logged %d times, want once per divergence shape", got) + } + if st := s.Stats(); st.Mismatches != 5 || st.Observed != 5 { + t.Fatalf("stats = %+v", st) + } +} + +func TestShadowDivergenceLogKindsAreBounded(t *testing.T) { + s, logs := newTestShadow(10) + for i := range maxLoggedDivergenceKinds + 20 { + s.Observe(brokerRequest([]string{"OWNERS"}, testGuards), Verdict{Effect: EffectAllow, ReasonCode: fmt.Sprintf("r%d", i)}) + } + if got := strings.Count(logs.String(), "authz shadow"); got != maxLoggedDivergenceKinds { + t.Fatalf("logged %d lines, want the bound %d", got, maxLoggedDivergenceKinds) + } + if st := s.Stats(); st.Mismatches != maxLoggedDivergenceKinds+20 { + t.Fatalf("every divergence must still be counted: %+v", st) + } +} + +type recordingSink struct { + mu sync.Mutex + events []Event +} + +func (r *recordingSink) Record(ev Event) { + r.mu.Lock() + defer r.mu.Unlock() + r.events = append(r.events, ev) +} + +type panickingSink struct{} + +func (panickingSink) Record(Event) { panic("sink bug") } + +func TestShadowSinkReceivesEventsAndItsPanicIsContained(t *testing.T) { + s, _ := newTestShadow(4) + sink := &recordingSink{} + s.Sink = sink + s.Observe(brokerRequest([]string{"a.go"}, testGuards), legacyAllow) + if len(sink.events) != 1 || sink.events[0].DecisionID != s.Events()[0].DecisionID { + t.Fatalf("sink events = %+v", sink.events) + } + + s.Sink = panickingSink{} + s.Observe(brokerRequest([]string{"a.go"}, testGuards), legacyAllow) + if st := s.Stats(); st.Errors != 1 || st.Observed != 2 || len(s.Events()) != 2 { + t.Fatalf("a sink panic must be counted and contained: %+v", st) + } +} + +func TestShadowZeroValueDoesNotPanic(t *testing.T) { + s := &Shadow{} + s.Observe(brokerRequest([]string{"a.go"}, testGuards), legacyAllow) + if got := s.Events(); got != nil { + t.Fatalf("a Shadow without a stream recorded %d events", len(got)) + } +} + +func TestShadowConcurrentObserve(t *testing.T) { + const workers, perWorker, capacity = 8, 50, 16 + s, _ := newTestShadow(capacity) + var wg sync.WaitGroup + for range workers { + wg.Add(1) + go func() { + defer wg.Done() + for range perWorker { + s.Observe(brokerRequest([]string{"a.go"}, testGuards), legacyAllow) + _ = s.Events() + _ = s.Stats() + } + }() + } + wg.Wait() + st := s.Stats() + if st.Observed != workers*perWorker || st.Buffered != capacity || st.Dropped != workers*perWorker-capacity { + t.Fatalf("stats = %+v", st) + } +} + +func TestDefaultShadowIsStable(t *testing.T) { + if DefaultShadow() != DefaultShadow() { + t.Fatal("DefaultShadow must return the same instance for the life of the process") + } +} + +func TestDecisionIDsAreUnique(t *testing.T) { + seen := map[string]struct{}{} + for range 100 { + id := newDecisionID() + if _, dup := seen[id]; dup { + t.Fatalf("duplicate decision id %q", id) + } + seen[id] = struct{}{} + } +} diff --git a/src/crux/pushbroker/protected.go b/src/crux/pushbroker/protected.go index 17258d7387..a601b2d7a1 100644 --- a/src/crux/pushbroker/protected.go +++ b/src/crux/pushbroker/protected.go @@ -6,6 +6,8 @@ import ( "fmt" "path/filepath" "strings" + + "github.com/hivecommons/hive/crux/authz" ) // guardChangedFiles records the outgoing file list on res and refuses an @@ -16,16 +18,39 @@ func (b *Broker) guardChangedFiles(ctx context.Context, res *Result) error { return err } res.ChangedFiles = files + protected := b.protectedPaths() if len(files) == 0 { + b.observeChangedFiles(files, protected, authz.Verdict{Effect: authz.EffectDeny, ReasonCode: authz.ReasonEmptyPush}) return errors.New("pushbroker: no committed changes to push") } - if rejected := ProtectedPathViolations(files, b.protectedPaths()); len(rejected) > 0 { + if rejected := ProtectedPathViolations(files, protected); len(rejected) > 0 { + b.observeChangedFiles(files, protected, authz.Verdict{Effect: authz.EffectDeny, ReasonCode: authz.ReasonProtectedPath}) res.ProtectedReject = rejected return fmt.Errorf("pushbroker: protected paths changed: %s", strings.Join(rejected, ", ")) } + b.observeChangedFiles(files, protected, authz.Verdict{Effect: authz.EffectAllow, ReasonCode: authz.ReasonOK}) return nil } +// observeChangedFiles reports this guard's verdict to the authz shadow. It is +// called after the guard has decided and its result is not read back: the +// guard above stays the only authority (RFC #18988 phase 0/1). +func (b *Broker) observeChangedFiles(files, protected []string, legacy authz.Verdict) { + shadow := b.Shadow + if shadow == nil { + shadow = authz.DefaultShadow() + } + if shadow == nil { + return + } + shadow.Observe(authz.Request{ + Actor: authz.Actor{Agent: b.AgentName, Identified: strings.TrimSpace(b.AgentName) != ""}, + Action: authz.Action{Kind: authz.KindBranchPush, Surface: authz.SurfaceBroker, Forge: authz.ForgeGitHub}, + Resource: authz.Resource{Repo: b.Repo, Branch: b.Branch, Paths: files}, + Context: authz.Context{ProtectedPaths: protected}, + }, legacy) +} + func (b *Broker) protectedPaths() []string { if len(b.ProtectedPaths) > 0 { return b.ProtectedPaths diff --git a/src/crux/pushbroker/pushbroker.go b/src/crux/pushbroker/pushbroker.go index ee61b7ca5f..fe80b39f9b 100644 --- a/src/crux/pushbroker/pushbroker.go +++ b/src/crux/pushbroker/pushbroker.go @@ -15,6 +15,7 @@ import ( "strings" "time" + "github.com/hivecommons/hive/crux/authz" "github.com/hivecommons/hive/crux/effects" "github.com/hivecommons/hive/crux/gitidentity" "github.com/hivecommons/hive/crux/logscrub" @@ -131,6 +132,11 @@ type Broker struct { Logger *slog.Logger Now func() time.Time Mutation effects.Boundary + // Shadow, when set, observes the changed-files guard in shadow mode + // (RFC #18988 phase 0/1): the authz evaluator is compared with this + // guard's own verdict and an event is recorded. It enforces nothing. Nil + // uses authz.DefaultShadow(), which is off unless HIVE_AUTHZ_SHADOW is true. + Shadow *authz.Shadow // LeaseSHA, when set, makes the push an update of an EXISTING branch: a // hive-opened PR's head that a sandboxed kick checked out // (hivecommons/hive#18574). The outgoing commits must descend from it, diff --git a/src/crux/pushbroker/pushbroker_shadow_test.go b/src/crux/pushbroker/pushbroker_shadow_test.go new file mode 100644 index 0000000000..1a135ab2f1 --- /dev/null +++ b/src/crux/pushbroker/pushbroker_shadow_test.go @@ -0,0 +1,140 @@ +package pushbroker + +import ( + "context" + "errors" + "log/slog" + "reflect" + "slices" + "testing" + "time" + + "github.com/hivecommons/hive/crux/authz" +) + +const ( + shadowDiffTreeKey = "diff-tree --root --no-commit-id --no-renames --name-only -r -z HEAD" + shadowShowKey = "show --text --no-textconv --format= --no-ext-diff HEAD" +) + +func shadowScriptedGit(changed string) *scriptedGit { + return &scriptedGit{ + replies: map[string]string{ + "rev-parse HEAD": "abc123\n", + shadowDiffTreeKey: changed, + shadowShowKey: "+// harmless\n", + }, + fails: map[string]error{"rev-parse --verify refs/remotes/origin/work": errors.New("unknown revision")}, + } +} + +func shadowBroker(t *testing.T, git *scriptedGit, shadow *authz.Shadow) *Broker { + t.Helper() + fixed := time.Date(2026, 11, 18, 12, 0, 0, 0, time.UTC) + return &Broker{ + Workspace: fakeGitWorkspace(t), Branch: "work", Repo: "hivecommons/hive", AgentName: "scanner", + Minter: fakeMinter{"ghs_tok"}, Runner: git, Logger: slog.New(slog.DiscardHandler), + Now: func() time.Time { return fixed }, Shadow: shadow, + } +} + +// The authz shadow observes the changed-files guard without changing it: for +// each outcome the guard reaches, exactly one event is recorded and it agrees +// with the guard. +func TestBrokerShadowRecordsTheChangedFilesGuard(t *testing.T) { + cases := []struct { + name string + changed string + wantErr bool + wantEffect authz.Effect + wantReason string + wantPaths int + }{ + {"push passes", "pkg/safe.go\x00", false, authz.EffectAllow, authz.ReasonOK, 1}, + {"protected path refused", "policies/x.rego\x00pkg/safe.go\x00", true, authz.EffectDeny, authz.ReasonProtectedPath, 2}, + {"workflow refused", ".github/workflows/ci.yml\x00", true, authz.EffectDeny, authz.ReasonProtectedPath, 1}, + {"nothing committed refused", "", true, authz.EffectDeny, authz.ReasonEmptyPush, 0}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + shadow := authz.NewShadow(8) + _, err := shadowBroker(t, shadowScriptedGit(tc.changed), shadow).Run(context.Background()) + if (err != nil) != tc.wantErr { + t.Fatalf("Run error = %v, wantErr %v", err, tc.wantErr) + } + events := shadow.Events() + if len(events) != 1 { + t.Fatalf("events = %d, want one per guard decision", len(events)) + } + ev := events[0] + if ev.Effect != string(tc.wantEffect) || ev.ReasonCode != tc.wantReason || ev.Parity != authz.ParityMatch || !ev.Shadow { + t.Fatalf("event = %+v", ev) + } + if ev.Surface != string(authz.SurfaceBroker) || ev.Action.Kind != authz.KindBranchPush || ev.Resource.Repo != "hivecommons/hive" || + ev.Resource.Branch != "work" || ev.Actor.Agent != "scanner" || ev.Resource.PathsCount != tc.wantPaths { + t.Fatalf("event attributes = %+v", ev) + } + if st := shadow.Stats(); st.Mismatches != 0 || st.Errors != 0 { + t.Fatalf("stats = %+v, want no divergence from the existing guard", st) + } + }) + } +} + +// Enforcing nothing is the contract: the Result, the error and the git +// commands issued are identical with the shadow on and off. +func TestBrokerShadowChangesNoBehaviour(t *testing.T) { + for _, changed := range []string{"pkg/safe.go\x00", "OWNERS\x00", ""} { + gitOff, gitOn := shadowScriptedGit(changed), shadowScriptedGit(changed) + resOff, errOff := shadowBroker(t, gitOff, nil).Run(context.Background()) + resOn, errOn := shadowBroker(t, gitOn, authz.NewShadow(4)).Run(context.Background()) + if (errOff == nil) != (errOn == nil) || (errOff != nil && errOff.Error() != errOn.Error()) { + t.Fatalf("changed=%q: error differs with the shadow on: %v vs %v", changed, errOff, errOn) + } + // Workspaces are distinct temp dirs; compare everything else. + resOff.Workspace, resOn.Workspace = "", "" + if !reflect.DeepEqual(resOff, resOn) { + t.Fatalf("changed=%q: Result differs with the shadow on:\n off=%+v\n on=%+v", changed, resOff, resOn) + } + if !slices.Equal(gitOff.calls, gitOn.calls) || gitOff.pushed != gitOn.pushed { + t.Fatalf("changed=%q: git calls differ with the shadow on:\n off=%v\n on=%v", changed, gitOff.calls, gitOn.calls) + } + } +} + +// With no Shadow set and HIVE_AUTHZ_SHADOW unset, nothing is recorded anywhere. +func TestBrokerShadowOffRecordsNothing(t *testing.T) { + if authz.DefaultShadow() != nil { + t.Skip("HIVE_AUTHZ_SHADOW is set in this environment") + } + b := shadowBroker(t, shadowScriptedGit("OWNERS\x00"), nil) + if _, err := b.Run(context.Background()); err == nil { + t.Fatal("expected the protected path refusal") + } + if got := authz.DefaultShadow().Events(); got != nil { + t.Fatalf("off recorded %d events", len(got)) + } +} + +// A custom protected list reaches the evaluator, and its version differs from +// the default list's, so an event says which policy input it used. +func TestBrokerShadowSeesTheEffectiveProtectedList(t *testing.T) { + versions := map[string]string{} + for name, protected := range map[string][]string{"default": nil, "custom": {"docs/"}} { + shadow := authz.NewShadow(4) + b := shadowBroker(t, shadowScriptedGit("docs/notes.md\x00"), shadow) + b.ProtectedPaths = protected + _, err := b.Run(context.Background()) + events := shadow.Events() + if len(events) != 1 || events[0].Parity != authz.ParityMatch { + t.Fatalf("%s: events = %+v", name, events) + } + versions[name] = events[0].PolicyVersion + if wantErr := name == "custom"; (err != nil) != wantErr { + t.Fatalf("%s: Run error = %v", name, err) + } + } + if versions["default"] == versions["custom"] { + t.Fatalf("policy version did not reflect the protected list: %v", versions) + } +} diff --git a/src/docs/design/README.md b/src/docs/design/README.md index a2f86b66a7..acdf26681f 100644 --- a/src/docs/design/README.md +++ b/src/docs/design/README.md @@ -66,7 +66,7 @@ that status is the thing to check before treating a page as current behaviour: - [Crux: generic tool policy for the MITM proxy](crux-tool-policy.md) — **Design only (proposed).** #14228, child of #14138: a shared operation vocabulary with per-forge adapters, a `relay()` verdict, deny-by-default for unrecognised operations, and the proxy's evaluation pipeline written down once. It also covers migrating behind a parity test and treating #14211 as the first non-GitHub adapter. -- [One authorization point: decision seam, decision events and scoped credentials](authorization-decision-point.md) - **Design only (proposed).** RFC #18988: inventories today's write-side authorization decisions (proxy mode table and deny rules, push broker checks, relay authorizers, tool-approval CEL rules, mint and installation tokens, audit) with file:line citations, shows where they disagree, and proposes one evaluation seam, CEL narrowing rules for operators, one decision event per decision, fail-closed semantics per effect, and a five-phase shadow-first migration with named parity tests. +- [One authorization point: decision seam, decision events and scoped credentials](authorization-decision-point.md) - **Partly shipped** (phase 1 types and a shadow-mode evaluator for one gate, the push broker's changed-files guard, off by default and enforcing nothing; the rest is proposed). RFC #18988: inventories today's write-side authorization decisions (proxy mode table and deny rules, push broker checks, relay authorizers, tool-approval CEL rules, mint and installation tokens, audit) with file:line citations, shows where they disagree, and proposes one evaluation seam, CEL narrowing rules for operators, one decision event per decision, fail-closed semantics per effect, and a five-phase shadow-first migration with named parity tests. - [PR follow-up session resume](pr-follow-up-resume.md) - **shipped (v5, default off).** #9583: ties each agent-opened PR to the CLI session that authored it (a `pkg/turn` envelope per PR) and feeds CI failures, changes-requested reviews, new review-bot threads and comments from people with write access back into that session without `/clear` while it is still live. When the session is gone (a restart or the next regular kick), the follow-up takes the fix-before-new path and the agent's next kick carries a compact handoff note of the PR's original reasoning. Pointers are pruned on merge or close; a live-review soak is the remaining follow-up. diff --git a/src/docs/design/authorization-decision-point.md b/src/docs/design/authorization-decision-point.md index 55c8ac197c..2717f03c89 100644 --- a/src/docs/design/authorization-decision-point.md +++ b/src/docs/design/authorization-decision-point.md @@ -1,9 +1,11 @@ # One authorization point: decision seam, decision events and scoped credentials (#18988) -Status: **Proposed (design only).** Spec for RFC -[#18988](https://github.com/hivecommons/hive/issues/18988). No code or behaviour -changes with this record. The RFC stays open until its acceptance criteria are -met; this page is the concrete proposal to review against them. Related records: +Status: **Partly shipped (shadow mode only).** Spec for RFC +[#18988](https://github.com/hivecommons/hive/issues/18988). The first slice of +the migration is in the tree, off by default and enforcing nothing: see +[Phase status](#phase-status). Everything else on this page is still a +proposal. The RFC stays open until the enforcement phases are decided; this +page is the concrete proposal to review against its acceptance criteria. Related records: [crux-tool-policy.md](crux-tool-policy.md) (the proxy's own operation vocabulary, a narrower scope than this page), [tool-approval-desk.md](tool-approval-desk.md) (the one declarative rule layer that exists today), [crux.md](crux.md) (the @@ -539,6 +541,44 @@ pure functions of the workspace), then proxy hard denies and write-surface, then proxy REST mode table, then relays, then GraphQL (decision parity with the REST path is verified first, see open questions). +### Phase status + +| Phase | Status | +|---|---| +| 1. Types and schema | **Shipped for one gate.** `src/crux/authz` has `Request`, `Decision`, the `v1` event, and a rule id and reason code for the push broker's changed-files guard only. The registry for the other gates is not written | +| 2. Decision events behind a flag | **Started, one gate.** With `HIVE_AUTHZ_SHADOW=true` the push broker's changed-files guard emits one event per call to a bounded in-memory stream (`HIVE_AUTHZ_SHADOW_BUFFER`, default 1000, evictions counted). The proxy, the relays, the rest of the broker and the volume probe are not done, and there is no persistent stream or sink wiring yet, so the volume question stays open | +| 3. Seam in shadow | **Started, one gate.** `authz.Decide` re-derives the broker's changed-files decision after the guard has decided, and a difference is counted and logged as a divergence. Parity is covered by `TestParityPushBrokerChangedFiles` and `TestParityPushBrokerPathShapes` over the broker's existing cases. The proxy mode table, GraphQL, Linear, the relays and tool approval have no evaluator yet | +| 4. Enforce through the seam | Not started. Nothing reads a `Decision` to allow or refuse an action | +| 5. Operator rules and credentials | Not started | + +What this slice does and does not do, as built: + +- **Default off.** Unset, empty or unparseable `HIVE_AUTHZ_SHADOW` is off. Off + records nothing and allocates nothing: the broker holds a nil recorder. +- **Enforces nothing.** The broker calls the shadow after the guard has + returned its verdict and never reads a result back. A recorder panic is + recovered and counted. A test pins that the broker's `Result`, error and git + commands are identical with the shadow on and off. +- **Fail closed is written down, not applied.** `authz.Decide` already returns + a deny with `authz_error` for an evaluator panic, `missing_attribute` for a + missing repo or branch, and `unsupported_action` for an action with no rule, + and tests pin that. In this phase a shadow deny that the guard did not issue + is a divergence log line only. +- **Independent mirror.** The evaluator's path matching is written separately + from `pushbroker.ProtectedPathViolations`, so drift between the two is + caught by the parity tests and by the live divergence counter rather than + hidden by a shared function. +- **Attributes only.** An event carries the agent name, repo, branch, a count + of changed paths and a count of matched paths. Free-text attributes are + scrubbed for credential shapes, stripped of control characters and cut to 256 + bytes. No path list, body, diff or credential is in the event type. +- **Policy version.** The event's `policy_version` is the floor version plus a + short digest of the protected path list in force, so a per-call override of + the list is visible in the record. +- **Not yet an audit trail.** The stream is in memory, bounded and lost on + restart. It exists to measure parity. The dedicated durable stream and its + integrity are phase 2 work and open question 6. + Proposed task-sized issues (not filed; for the RFC's acceptance criteria): - `authz`: request, decision and event types plus the rule id registry (phase 1) diff --git a/src/docs/documentation-map.md b/src/docs/documentation-map.md index 661b70b1d3..8a848e1614 100644 --- a/src/docs/documentation-map.md +++ b/src/docs/documentation-map.md @@ -253,7 +253,7 @@ Documentation for the current Hive line (branch `v5`; the code and docs live und - [Session taint design](design/session-taint.md) - **design/proposal:** a session-level record of what an agent has read and the rules that would hold or deny later pushes, comments and PRs against it, off by default with an observe-only first phase (RFC #18987). - [Gate-integrity invariants for agent lanes](design/gate-integrity-invariants.md) — proposed write-gate rules for agent lanes: no history rewrites on branches a lane did not create, no sign-off on other authors' commits, and hold-gated demotion for gate manipulation. - [Crux: the agent runtime boundary](design/crux.md) — **Design record:** runtime separation, module boundary, staged rollout, and naming decision. -- [One authorization point for writes](design/authorization-decision-point.md) - proposed design for a single decision seam, decision events and per-decision scoped credentials across the proxy, relays and push broker (RFC #18988). +- [One authorization point for writes](design/authorization-decision-point.md) - design for a single decision seam, decision events and per-decision scoped credentials across the proxy, relays and push broker (RFC #18988); the first slice, a shadow-mode check of the push broker's changed-files guard, is shipped, off by default and enforces nothing. - [Customers: a profile per person and a customer graph](design/customers.md) — proposed design for the Customers section, the people store, the read-only graph endpoint, and inferred goals and satisfaction (RFC #18742). - [Design documents](design/README.md) — browse records labelled shipped, partly shipped, design-only, or historical; proposals are not current behavior. - [Discord reaction-consensus issue promotion](design/discord-issue-promotion.md) — proposed design for turning Discord reaction consensus into an audited Hive label write that composes with `project.issue_filter.require_labels` (RFC #6239). diff --git a/src/docs/env-vars.md b/src/docs/env-vars.md index 7e60f0c4ad..1a3def9f04 100644 --- a/src/docs/env-vars.md +++ b/src/docs/env-vars.md @@ -274,6 +274,8 @@ new value at the same time. | `HIVE_PROXY_ADVISORY_OK` | No | `false` | Allows the spoke to start when the forced-proxy egress redirect cannot be installed (no `CAP_NET_ADMIN`/iptables). Enforcement becomes advisory-only — agents can bypass the proxy. Also gates whether the Go proxy trusts a self-asserted `Proxy-Authorization` header as agent identity when its UID map is unavailable (N7, #3841) — off by default, an unidentified caller is treated as `ADVISORY` (writes blocked) rather than whatever name it claims. See [security-model.md](security-model.md#5-network-and-credentials). | | `HIVE_PROXY_ALLOW_PRIVATE_FORWARD` | No | `false` | Lifts the proxy's destination guard ([#14082](https://github.com/hivecommons/hive/issues/14082), [#16785](https://github.com/hivecommons/hive/pull/16785)). Off by default: a plain-HTTP (non-CONNECT) forward, a CONNECT to a host the proxy does not inspect, and a transparent-TLS passthrough are each dialled only when the resolved address is globally routable, so an agent cannot use the proxy to reach loopback services beside it, the link-local cloud metadata endpoint (`169.254.169.254`), RFC 1918/CGNAT cluster addresses or multicast, and the refusal is a `403` naming the class. Only the exact string `true` turns it on, and it lifts the guard for all three legs at once, for deployments that genuinely egress to a private network through the proxy (a LAN mirror on a native install); the proxy logs a boot-time warning when it is set. Loopback refusals stay at `DEBUG` and are not counted as violations, because agents are already handed `NO_PROXY` for loopback (#14076). With the guard lifted, the inference translator also stops trusting the proxy UID (`dev`, 1001 on v5 pods): agents that run as the hive user without a per-agent UID are refused with `reason=proxy_uid_private_forward` ([#18061](https://github.com/hivecommons/hive/issues/18061), [#14448](https://github.com/hivecommons/hive/issues/14448)). | | `HIVE_PROXY_INJECT_GH_AUTH` | No | unset (off on every hive, including hosted) | Exact `true` enables proxy-side App-token injection. **Requires GitHub App auth. Refuses startup (exit 19) with `HIVE_PROXY_ADVISORY_OK=true`.** Other spellings do not enable it. Read [behavior and exceptions](#proxy-credential-injection). | +| `HIVE_AUTHZ_SHADOW` | No | unset (off) | Parseable true value (`true`, `1`) turns on the authorization decision seam in shadow mode ([design](design/authorization-decision-point.md), RFC #18988 phase 0/1): the push broker's changed-files guard is re-evaluated by `authz.Decide` after it decides, one decision event per call is kept on a bounded in-memory stream and a difference from the guard is logged. It enforces nothing and changes no behaviour. Read once at first use, so a change needs a restart. | +| `HIVE_AUTHZ_SHADOW_BUFFER` | No | `1000` | Number of decision events the shadow stream keeps (oldest evicted first, evictions counted). Positive integer, capped at `10000`; anything else uses the default. Only read when `HIVE_AUTHZ_SHADOW` is on. | | `HIVE_DEPLOYMENT_RUNTIME` | No | none (config `deployment.runtime` fallback; hub-registered Kubernetes hives are auto-detected) | Declares the deployment runtime (`kubernetes`, `podman-quadlet`, or `docker-compose`) so the dashboard can offer standalone upgrades. Unset/unrecognized means `unknown` and the upgrade button stays hidden. See [dashboard-standalone-upgrades.md](dashboard-standalone-upgrades.md). | | `HIVE_DEPLOYMENT_PODMAN_MODE` | Only when runtime is `podman-quadlet` | none (config `deployment.podman_mode` fallback) | `rootless` (`systemctl --user`) or `rootful` (system manager). Without an explicit mode the Podman/Quadlet upgrade path stays disabled. | | `HIVE_DASHBOARD_UPGRADE_HELPER` | No | config `deployment.upgrade_helper` fallback, then `/usr/local/libexec/hive-dashboard-upgrade-helper` | Absolute path to the standalone upgrade helper executable the dashboard invokes. Must be a regular, executable file at an absolute path or the upgrade button stays hidden. See [dashboard-standalone-upgrades.md](dashboard-standalone-upgrades.md). | diff --git a/src/docs/security-model.md b/src/docs/security-model.md index 2ca84a5d44..b10167e7c1 100644 --- a/src/docs/security-model.md +++ b/src/docs/security-model.md @@ -70,6 +70,7 @@ are enabled on a given hive. | `ioscan` scanning and canaries (default on) | prompt injection, secret leakage | `src/pkg/ioscan` | `ioscanEnabled`, `ioscanFailMode` in `GET /api/config/governor` | | Trajectory review (default on) | goal drift | `src/crux/trajectory` | `governor.trajectory.enabled` | | Policy proxy and repo allowlist | credential theft, out-of-scope writes | `src/crux/proxy/rules.go` | `403` with `X-Hive-Proxy-Blocked` | +| Authorization decision seam, shadow mode (opt-in, enforces nothing) | undetected drift between a gate and its declared policy | `src/crux/authz`, `HIVE_AUTHZ_SHADOW` | `authz shadow` log lines; no behaviour change either way | | Forced egress through the proxy (iptables) | proxy bypass | `src/deploy/entrypoint.sh` | boot log; no `HIVE_PROXY_ADVISORY_OK` | | Tiered GitHub tokens per agent mode | blast radius of a leaked token | `src/crux/agentmode`, `src/crux/agent/manager_tokens.go` | token tier on the agent card | | Proxy-side credential injection (opt-in) | token theft | `HIVE_PROXY_INJECT_GH_AUTH` | `security.credentialInjection` in governor config | @@ -386,6 +387,28 @@ telemetry only: health, version, agent and governor summaries and 24-hour token counts. No credential leaves a spoke in a heartbeat. The hub delivers configuration, including App credentials for hosted spokes, in the response. +**Authorization decision point (shadow mode).** Several gates decide whether +a write may happen. They sit in different places, and the +[design record](design/authorization-decision-point.md) proposes one shared +point for them. The first slice is a measurement, and it is off by default. +Set `HIVE_AUTHZ_SHADOW=true` to turn it on. + +With it on, a small checker re-derives one existing decision. That decision is +the push broker's check for an empty push or a protected path. The checker +runs after the broker has already decided. Hive compares the two answers and +keeps one short event for each check. It logs any difference. + +The shadow enforces nothing. The broker stays the only authority, so a push +goes through or is refused exactly as before. Nothing acts on the checker's +answer. An event holds names, counts and codes. It never holds a request +body, a diff, a file, a list of paths or a credential. Free text in an event is +scrubbed and cut to a fixed length. + +The seam is meant to fail closed once it enforces, and that rule is already +written down and tested. An error or crash in the checker counts as a deny. A +request with a missing required field also counts as a deny. In this phase +those are only log lines, not refusals. + ## 6. Code admission A green check is necessary but not sufficient. Between green CI and a merge, From f49671dc1478e49bb569dfc9aa308518b5a39dd7 Mon Sep 17 00:00:00 2001 From: Andrew Anderson Date: Wed, 18 Nov 2026 14:19:18 -0500 Subject: [PATCH 2/3] =?UTF-8?q?=F0=9F=90=9B=20authz:=20satisfy=20staticche?= =?UTF-8?q?ck=20and=20refresh=20shifted=20docs=20citations?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Refs #18988 Signed-off-by: Andrew Anderson --- src/crux/authz/shadow_test.go | 4 +++- src/docs/design/session-taint.md | 4 ++-- 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/src/crux/authz/shadow_test.go b/src/crux/authz/shadow_test.go index 899b4123cd..3f6e9c2153 100644 --- a/src/crux/authz/shadow_test.go +++ b/src/crux/authz/shadow_test.go @@ -338,7 +338,9 @@ func TestShadowConcurrentObserve(t *testing.T) { } func TestDefaultShadowIsStable(t *testing.T) { - if DefaultShadow() != DefaultShadow() { + first := DefaultShadow() + second := DefaultShadow() + if first != second { t.Fatal("DefaultShadow must return the same instance for the life of the process") } } diff --git a/src/docs/design/session-taint.md b/src/docs/design/session-taint.md index 478aacfc93..8fc26b4ff3 100644 --- a/src/docs/design/session-taint.md +++ b/src/docs/design/session-taint.md @@ -81,7 +81,7 @@ has strong per-message checks and no memory. | Canary at the proxy | `inspectCanaryEgress` (`src/crux/proxy/github_proxy_policy.go:266`), called at `src/crux/proxy/github_proxy_mitm.go:242` | Reads write bodies up to `maxGitHubWriteBodyScan` and denies with code `DenyReasonCanaryEgress` (`src/crux/proxy/denial_message.go:24`) in closed mode. Refuses `git-receive-pack` in closed mode because the pack cannot be scanned. | Only bodies on paths `githubWriteBodyMayLeak` accepts (`src/crux/proxy/github_proxy_policy.go:363`): issues, pulls, comments, reviews, git objects, GraphQL. | | Canary at the relays | `scanCanaryText` call sites at `src/pkg/github/issue_request_watcher.go:1596`, `src/pkg/github/pullrequest.go:233`, `src/pkg/github/review_request_watcher.go:598`; advisory findings at `src/cmd/hive/eval_cycle_seams.go:722` | Same scan on relay bodies and on agent advisory findings; leak recorded as `ioscan_canary_leak` with a critical bead by `handleCanaryLeak` (`src/cmd/hive/github_client_config.go:482`). | Same limit: marker only. | | Secret classifier | `FindSecret` (`src/crux/logscrub/handler.go:157`) over `secretPatterns` (`src/crux/logscrub/handler.go:37`) | Returns a category for the first exact-shape credential, never the value. Heuristic shapes are skipped on purpose "because refusing a push on one blocks legitimate work". | Exact shapes only. A token in an unknown format, or a reworded secret, is not found. | -| Push broker | The secret gate in `Broker.Run` (`src/crux/pushbroker/pushbroker.go:238`) | Rejects an outgoing diff containing a `FindSecret` hit, after fast-forward, protected-path and sign-off checks. The push itself is recorded through the effects boundary and the broker is built by the sandbox executor. | The scan covers the diff (`git diff`, or `git show` when no base resolves), not commit messages **(unverified beyond reading the two git invocations)**. It has no idea what the agent read. | +| Push broker | The secret gate in `Broker.Run` (`src/crux/pushbroker/pushbroker.go:244`) | Rejects an outgoing diff containing a `FindSecret` hit, after fast-forward, protected-path and sign-off checks. The push itself is recorded through the effects boundary and the broker is built by the sandbox executor. | The scan covers the diff (`git diff`, or `git show` when no base resolves), not commit messages **(unverified beyond reading the two git invocations)**. It has no idea what the agent read. | | Proxy modes and rules | `rules` (`src/crux/proxy/rules.go:135`), `AllowedByModeCaps` (`src/crux/proxy/rules.go:248`), `denyRules` (`src/crux/proxy/rules_deny.go:43`) | Method and path need a minimum ACMM mode; unknown writes are denied; direct PR create and merge are hard-denied. | Looks at the request, not at history. | | Write-surface gate | `WriteSurfaceEnforceRefusal` (`src/crux/proxy/write_surface_enforce.go:49`) | For a lane under `write_surface.enforce`, every direct write is refused so the relays are the only write path. | Says nothing about relay content. | | Which hosts are inspected | `NeedsMITM` (`src/crux/proxy/rules.go:89`), `IsCopilotAPIHost` (`src/crux/proxy/rules.go:124`) | TLS is terminated for the forge API host and the tracker host, and for completion hosts only to read usage. Everything else is an opaque tunnel (`tunnelDirect`, `src/crux/proxy/github_proxy_connect.go:605`) guarded by destination class (`plainForwardDenyReason`, `src/crux/proxy/plain_forward_guard.go:119`). | No payload visibility on other hosts, so no content-based egress rule is possible there. | @@ -172,7 +172,7 @@ agent, epoch)` and a pure function from `(labels, descriptor)` to a verdict. |---|---|---| | S-relay: `hive-open-issue`, `hive-open-pr`, `hive-review` and the spool-processed equivalents | beside `scanCanaryText` in `pkg/github` | Host-side, attribution comes from the manifest, so it works for sandboxed lanes without proxy identity. Highest value, lowest risk. | | S-proxy: direct REST and GraphQL writes by lanes not under `write_surface.enforce` | body-gate stage of the pipeline in [crux-tool-policy](crux-tool-policy.md), next to the canary gate | Requires UID attribution; observe-only when `HIVE_PROXY_ADVISORY_OK` is in effect. | -| S-push: the broker and any `git-receive-pack` | the broker, beside its secret gate and before the push token is minted (`src/crux/pushbroker/pushbroker.go:238`) | Destination visibility is the point: a push to a public fork with a `private-repo` label is the RFC flow 1 in code form. | +| S-push: the broker and any `git-receive-pack` | the broker, beside its secret gate and before the push token is minted (`src/crux/pushbroker/pushbroker.go:244`) | Destination visibility is the point: a push to a public fork with a `private-repo` label is the RFC flow 1 in code form. | | S-pr: the sandbox PR step (`CreateAgentPR` after a broker push) | `src/crux/agent/sandbox_executor.go` after `broker.Run` | PR title and body come from workspace files (`sandboxPRText`), so they are agent-authored text. | | S-advisory: agent advisory findings | the eval-cycle canary gate | Same text, same rule. | | S-egress: connections to hosts the proxy does not inspect | `tunnelDirect` | Destination-only: allow or hold by host, never content. Disabled by default even in enforce; see risks. | From 69cc1c5632df2c6e5fde99fbe5fa933154879572 Mon Sep 17 00:00:00 2001 From: Andrew Anderson Date: Wed, 18 Nov 2026 15:04:00 -0500 Subject: [PATCH 3/3] =?UTF-8?q?=F0=9F=93=96=20authz:=20refresh=20shifted?= =?UTF-8?q?=20pushbroker=20line=20citations=20in=20the=20design=20record?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Refs #18988 Signed-off-by: Andrew Anderson --- .../design/authorization-decision-point.md | 36 +++++++++---------- 1 file changed, 18 insertions(+), 18 deletions(-) diff --git a/src/docs/design/authorization-decision-point.md b/src/docs/design/authorization-decision-point.md index 2717f03c89..18c703b7f0 100644 --- a/src/docs/design/authorization-decision-point.md +++ b/src/docs/design/authorization-decision-point.md @@ -133,30 +133,30 @@ leaves no per-decision record** beyond the write counters. ### B. The push broker -`Broker.Run` (`src/crux/pushbroker/pushbroker.go:168-292`) runs a fixed +`Broker.Run` (`src/crux/pushbroker/pushbroker.go:174-300`) runs a fixed sequence, each step an early return through `fail` -(`src/crux/pushbroker/pushbroker.go:597-604`), which logs and returns the error: - -1. required fields (`validate`, `:296`); -2. hostile workspace config (`:175`, `src/crux/pushbroker/workspace_config.go:111`); -3. empty outgoing commits (`:183`); -4. fast-forward against remote and base (`:189-197`); -5. descend-from-lease when updating an existing PR branch (`:199`); -6. forged lane sign-off (`:203`, `:432`); -7. changed-files guard: empty push, then protected paths (`:207`, - `src/crux/pushbroker/protected.go:13-27`); -8. outgoing-diff secret scan (`:238`); -9. mint token (`:242`), then push inside `effects.Execute` with - `Kind: KindBranchPush` (`:277-282`). +(`src/crux/pushbroker/pushbroker.go:603-610`), which logs and returns the error: + +1. required fields (`validate`, `:302`); +2. hostile workspace config (`:181`, `src/crux/pushbroker/workspace_config.go:111`); +3. empty outgoing commits (`:189`); +4. fast-forward against remote and base (`:195-203`); +5. descend-from-lease when updating an existing PR branch (`:205`); +6. forged lane sign-off (`:209`, `:438`); +7. changed-files guard: empty push, then protected paths (`:213`, + `src/crux/pushbroker/protected.go:15-33`); +8. outgoing-diff secret scan (`:244`); +9. mint token (`:248`), then push inside `effects.Execute` with + `Kind: KindBranchPush` (`:283-288`). The protected-path list is `DefaultProtectedPaths` -(`src/crux/pushbroker/pushbroker.go:28-36`). `Broker.ProtectedPaths` overrides -it per call (`:128`, `src/crux/pushbroker/protected.go:29-34`), but the only +(`src/crux/pushbroker/pushbroker.go:29-37`). `Broker.ProtectedPaths` overrides +it per call (`:129`, `src/crux/pushbroker/protected.go:54-59`), but the only production construction, `src/crux/agent/sandbox_executor.go:377-389`, does not set it, and a grep found no config key feeding it. So the list is effectively a code constant today. **(unverified outside the files searched.)** -The broker records a `Result` (`src/crux/pushbroker/pushbroker.go:150-166`) +The broker records a `Result` (`src/crux/pushbroker/pushbroker.go:156-172`) with `ProtectedReject`, `SecretRejected` and `Error`. It carries no rule id, no policy version and no credential descriptor. The `effects.Claim` it submits has an `Actor` field (`src/crux/effects/effects.go:182-188`) that the broker leaves @@ -213,7 +213,7 @@ refusals as well as its successes is **unverified**. repositories (`src/pkg/github/app.go:291-345`, `:440-444`). The `contributor` tier carries issues, contents and pull-requests write (`src/pkg/github/app.go:331-337`). The broker uses `DefaultTier = "contributor"` - (`src/crux/pushbroker/pushbroker.go:25`), so a push token also carries issue + (`src/crux/pushbroker/pushbroker.go:26`), so a push token also carries issue and PR write that a push does not need. - **Callback mint.** `MintToken` returns a response with an `ExpiresAt` for the minted token (`src/pkg/cruxhost/cruxhost.go:220-241`). The relation