feat: add versioned declarative detection rules

.github/workflows/ci.yml

@@ -137,7 +137,11 @@ jobs:
           git diff --exit-code -- gen packages/connect/src/gen
           test -z "$(git status --porcelain -- gen packages/connect/src/gen)"
 
-      - run: go test ./...
+      # The bootstrap and ingestion-worker packages share this job's migrated
+      # Postgres database. Run their package binaries serially so a worker test
+      # cannot claim another package's queued fixture.
+      - name: Run database-backed Go tests
+        run: go test -p 1 ./...
 
       - name: Run bounded Go worker smokes
         run: npm run smoke:workers:go

CHANGELOG.md

@@ -4,12 +4,24 @@ All notable changes to Aperio are recorded here. Release entries are tied to a s
 
 ## [Unreleased]
 
+## [0.1.0] - 2026-08-21
+
+- Added tenant-scoped, hashed API tokens with read, write, and admin scopes,
+  expiry, revocation, last-used state, and audit records.
+- Added `aperioctl` commands for health checks, findings, connectors, sync,
+  SIEM destinations, and API-token lifecycle operations.
+- Added durable connector-sync and rule-run receipts plus authenticated
+  operator health for connector freshness, ingestion queues, SIEM delivery,
+  and recent rule execution.
+- Added a Prometheus endpoint that is disabled until a dedicated scrape token
+  is configured and does not expose tenant or resource labels.
+- Added a strict CEL/YAML detection engine with versioned rules for GitHub
+  public repositories, Slack MFA and external shared channels, and Google
+  Workspace external sharing.
+- Added tenant rule disablement, severity overrides, scoped auto-resolution,
+  an in-memory backtest API, and an explicit connector support matrix.
 - Added a locally-owned review preflight that checks workflow action pinning and reports required validation commands.
 - Removed vendor-backed review and code-writing workflows from the public repository.
 - Added a production Compose bundle with Postgres, NATS, API, web, ingestion, SIEM, and migration services.
 - Added release, upgrade, backup, security, and contributor documentation.
 - Updated the Node dependency lock and Go modules to patched release lines.
-
-## [0.1.0] - Not released
-
-Initial public product baseline. The first release date will be recorded when the signed image and source tag are published.

Makefile

@@ -289,7 +289,7 @@ test-go: ## Run Go unit tests
 .PHONY: test-go-db
 test-go-db: require-env ## Run Go tests including DB-backed routes (needs Postgres)
 	@$(MAKE) --no-print-directory db-up migrate
-	@$(LOAD_ENV) APERIO_TEST_DATABASE_URL="$$(node $(DEV_CONFIG) go-database-url)" go test ./...
+	@$(LOAD_ENV) APERIO_TEST_DATABASE_URL="$$(node $(DEV_CONFIG) go-database-url)" go test -p 1 ./...
 
 .PHONY: test-api
 test-api: require-env ## Run the TypeScript/node test suite

docs/detection-support-matrix.md

@@ -0,0 +1,52 @@
+# Detection support matrix
+
+This matrix describes the payload fields the current Go ingestion worker can
+actually evaluate. A connector is not treated as rule-supported merely because
+credentials can be stored; the provider must enqueue one of the listed event
+types with the listed fields.
+
+| Provider | Rule | Version | Event types | Required payload fields | Auto-resolution input | Default | Notes |
+| --- | --- | --- | --- | --- | --- | --- | --- |
+| GitHub | `github.public_repository_created` | `1.0.0` | `PUBLIC_REPOSITORY_CREATED`, `REPOSITORY_PUBLICIZED` (aliases accepted) | `repository.full_name`; `repository.visibility` or `repository.private` | `REPOSITORY_PRIVATE`, `REPOSITORY_PRIVATEIZED`, or `REPOSITORY_VISIBILITY_CHANGED` with private visibility | On | Declarative YAML pack; the finding target is the repository and the dedupe subject is the same repository. |
+| GitHub | `github.branch_protection_disabled` | `catalog` | `BRANCH_PROTECTION_DISABLED`, `BRANCH_PROTECTION_RULE_DELETED`, `BRANCH_PROTECTION_RULE_UPDATED` | repository name; branch/ref/rule pattern when available | Not currently automatic | On | Hardcoded compatibility rule. Updated rules fire only when the payload indicates weakened settings. |
+| GitHub | `github.oauth_app_installed` | `catalog` | `OAUTH_APP_INSTALLED`, `GITHUB_APP_INSTALLED`, `ORG_OAUTH_APP_ACCESS_APPROVED` | app name or ID; scopes/permissions when available | Not currently automatic | On | Payloads with no scope list are retained for review; known low-risk scoped installs are skipped. |
+| GitHub | `github.deploy_key_added` | `1.0.0` catalog | `DEPLOY_KEY_ADDED`, `DEPLOY_KEY_CREATED` (aliases accepted) | repository name; key title/name/ID; `key.write_enabled` when available | Not currently automatic | Off | Existing connector catalog exposes this opt-in check. Write-enabled keys escalate to HIGH; missing write metadata remains MEDIUM. |
+| Slack | `slack.mfa_disabled` | `1.0.0` | `MFA_DISABLED`, `TWO_FACTOR_AUTH_DISABLED` (aliases accepted) | `user.email` or `user.id` | `MFA_ENABLED` or `TWO_FACTOR_AUTH_ENABLED` with the same user field | On | Declarative YAML pack. The clean event is never treated as a disablement. |
+| Slack | `slack.external_shared_channel_created` | `1.0.0` | `EXTERNAL_SHARED_CHANNEL_CREATED`, `SHARED_CHANNEL_INVITE_ACCEPTED` | channel name/ID; external organization/team name | Not currently automatic | On | Declarative YAML pack; the finding is emitted only when both channel and external-organization identity are present. |
+| Slack | `slack.workspace_invite_link_enabled` | `catalog` | `WORKSPACE_INVITE_LINK_ENABLED`, `INVITE_LINK_CREATED` | workspace/team name when available | Not currently automatic | On | Hardcoded compatibility rule. |
+| Slack | `slack.app_installed` | `catalog` | `APP_INSTALLED`, `APP_APPROVED`, `APP_SCOPES_APPROVED` | app name/ID; scopes when available | Not currently automatic | Off | Existing rule escalates to HIGH when scopes include admin, file-history, or channel-history access; no claim is made when scope data is absent. |
+| Google Workspace | `google_workspace.external_sharing_enabled` | `1.0.0` | `EXTERNAL_SHARING_ENABLED` | `parameters.visibility`; document title/id/type/owner when available | `EXTERNAL_SHARING_DISABLED`, `DRIVE_FILE_VISIBILITY_CHANGED`, or `DRIVE_FILE_PRIVATE` with private/domain/internal visibility | On | Declarative YAML pack supports both resource metadata and Reports API parameter shapes. |
+
+## Tenant overrides
+
+The worker already reads `integration_connections.disabled_checks`. A severity
+override can be stored in the existing `disabled_check_metadata` JSON object
+without a new table:
+
+```json
+{
+  "slack.mfa_disabled": {
+    "reason": "tenant risk policy",
+    "severity": "HIGH",
+    "expiresAt": "2026-12-31T00:00:00Z"
+  }
+}
+```
+
+Only `CRITICAL`, `HIGH`, `MEDIUM`, `LOW`, and `INFO` are accepted. Expired or
+malformed entries are ignored. Disabling a rule still uses the existing
+`disabled_checks` array and its expiry behavior.
+
+## Known gaps
+
+- GitHub secret-scanning alerts, dependabot alerts, membership changes, and
+  webhook delivery are not advertised here because this checkout does not
+  currently normalize those provider payloads into supported ingestion event
+  types.
+- Slack message/file export volume, guest lifecycle, and user deactivation are
+  not advertised because the available payload contract does not guarantee the
+  required fields.
+- Rule efficacy rollups, persisted `rule_version` columns, community-pack
+  signatures, and stateful/correlation rules remain follow-up work. The
+  evaluator emits versioned drafts now so those consumers can be added without
+  changing rule semantics.

go.mod

@@ -3,23 +3,33 @@ module github.com/writer/aperio
 go 1.26.6
 
 require (
+	cel.dev/cel-go v0.32.0
 	connectrpc.com/connect v1.20.0
 	github.com/go-pdf/fpdf v0.9.0
 	github.com/jackc/pgx/v5 v5.10.0
 	github.com/nats-io/nats.go v1.53.1
 	github.com/writer/cerebro/sdk/go/cerebroapi v0.0.0-20260617190440-784f5eee34f2
 	golang.org/x/crypto v0.55.0
 	google.golang.org/protobuf v1.36.12
+	gopkg.in/yaml.v3 v3.0.1
 )
 
 require (
+	cel.dev/expr v0.25.1 // indirect
+	github.com/antlr4-go/antlr/v4 v4.13.1 // indirect
 	github.com/jackc/pgpassfile v1.0.0 // indirect
 	github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
 	github.com/jackc/puddle/v2 v2.2.2 // indirect
 	github.com/klauspost/compress v1.18.5 // indirect
+	github.com/kr/text v0.2.0 // indirect
 	github.com/nats-io/nkeys v0.4.15 // indirect
 	github.com/nats-io/nuid v1.0.1 // indirect
+	github.com/rogpeppe/go-internal v1.16.0 // indirect
+	go.yaml.in/yaml/v3 v3.0.4 // indirect
+	golang.org/x/exp v0.0.0-20240823005443-9b4947da3948 // indirect
 	golang.org/x/sync v0.22.0 // indirect
 	golang.org/x/sys v0.47.0 // indirect
 	golang.org/x/text v0.41.0 // indirect
+	google.golang.org/genproto/googleapis/api v0.0.0-20240826202546-f6391c0de4c7 // indirect
+	google.golang.org/genproto/googleapis/rpc v0.0.0-20240826202546-f6391c0de4c7 // indirect
 )

go.sum

@@ -1,5 +1,12 @@
+cel.dev/cel-go v0.32.0 h1:irvpFKr5EuGPyxeME03ERh0rii1TX+BDAnB9eL3IvNk=
+cel.dev/cel-go v0.32.0/go.mod h1:DnVip7tpJSsgZymwfT+m1tnEVy3ivAjSMXPx12YrMkU=
+cel.dev/expr v0.25.1 h1:1KrZg61W6TWSxuNZ37Xy49ps13NUovb66QLprthtwi4=
+cel.dev/expr v0.25.1/go.mod h1:hrXvqGP6G6gyx8UAHSHJ5RGk//1Oj5nXQ2NI02Nrsg4=
 connectrpc.com/connect v1.20.0 h1:6TNDAB+WeNd2uolWNlYczB5E0KNNaVMNUEx8JEUsPmQ=
 connectrpc.com/connect v1.20.0/go.mod h1:A2ygJrukXwWy32vkCAAHNVguZrqZ+jeZ9rGRnGR4dN4=
+github.com/antlr4-go/antlr/v4 v4.13.1 h1:SqQKkuVZ+zWkMMNkjy5FZe5mr5WURWnlpmOuzYWrPrQ=
+github.com/antlr4-go/antlr/v4 v4.13.1/go.mod h1:GKmUxMtwp6ZgGwZSva4eWPC5mS6vUAmOABFgjdkM7Nw=
+github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
 github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
 github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
 github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
@@ -17,6 +24,10 @@ github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo
 github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4=
 github.com/klauspost/compress v1.18.5 h1:/h1gH5Ce+VWNLSWqPzOVn6XBO+vJbCNGvjoaGBFW2IE=
 github.com/klauspost/compress v1.18.5/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
+github.com/kr/pretty v0.3.0 h1:WgNl7dwNpEZ6jJ9k1snq4pZsg7DOEN8hP9Xw0Tsjwk0=
+github.com/kr/pretty v0.3.0/go.mod h1:640gp4NfQd8pI5XOwp5fnNeVWj67G7CFk/SaSQn7NBk=
+github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
+github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
 github.com/nats-io/nats.go v1.53.1 h1:Otsq3uLc/kLdjmkNHkXH0jBqwUquwdKFoe3fq6/3/Xo=
 github.com/nats-io/nats.go v1.53.1/go.mod h1:26HypzazeOkyO3/mqd1zZd53STJN0EjCYF9Uy2ZOBno=
 github.com/nats-io/nkeys v0.4.15 h1:JACV5jRVO9V856KOapQ7x+EY8Jo3qw1vJt/9Jpwzkk4=
@@ -25,24 +36,36 @@ github.com/nats-io/nuid v1.0.1 h1:5iA8DT8V7q8WK2EScv2padNa/rTESc1KdnPw4TC2paw=
 github.com/nats-io/nuid v1.0.1/go.mod h1:19wcPz3Ph3q0Jbyiqsd0kePYG7A95tJPxeL+1OSON2c=
 github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
 github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
+github.com/rogpeppe/go-internal v1.16.0 h1:O9DK+vNMDVGLr2BeZqmpLeMjiMNkuXfcqntWbZV6S5g=
+github.com/rogpeppe/go-internal v1.16.0/go.mod h1:DrUVZyrJU+txYW5/1kwtXQSMFio52ZOxX7yM1VHvnxs=
 github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
 github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
 github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
 github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
 github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
 github.com/writer/cerebro/sdk/go/cerebroapi v0.0.0-20260617190440-784f5eee34f2 h1:+Jsrg6W9h3uUA8wt3N2hXtC3ZbiVVbvrPdIR01ClEXQ=
 github.com/writer/cerebro/sdk/go/cerebroapi v0.0.0-20260617190440-784f5eee34f2/go.mod h1:dHVu/CuhHRejAdS2yHB7oBCnfeT9ZVD4yxDHdPPzmhw=
+go.yaml.in/yaml/v3 v3.0.4 h1:tfq32ie2Jv2UxXFdLJdh3jXuOzWiL1fo0bu/FbuKpbc=
+go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
 golang.org/x/crypto v0.55.0 h1:+KWHjbgOaAQ66dh/YlkZKHlz9ZUlq61AFirAR9ntP8M=
 golang.org/x/crypto v0.55.0/go.mod h1:uq0V9dE/fzQuJtbnL+2EhWOE63vo164FY8xqEnV9xis=
+golang.org/x/exp v0.0.0-20240823005443-9b4947da3948 h1:kx6Ds3MlpiUHKj7syVnbp57++8WpuKPcR5yjLBjvLEA=
+golang.org/x/exp v0.0.0-20240823005443-9b4947da3948/go.mod h1:akd2r19cwCdwSwWeIdzYQGa/EZZyqcOdwWiwj5L5eKQ=
 golang.org/x/sync v0.22.0 h1:SZjpbeLmrCk4xhRSZFNZW5gFUeCeFgjekvI/+gfScek=
 golang.org/x/sync v0.22.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
 golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs=
 golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
 golang.org/x/text v0.41.0 h1:vz/seA0lnX87Othu2f/0L24RcgrXD9/YFTSuGjj3rH8=
 golang.org/x/text v0.41.0/go.mod h1:jvf1O8ajNzZqhSrQBPbutR/EB83Cc0CFrezNQIwbb5M=
+google.golang.org/genproto/googleapis/api v0.0.0-20240826202546-f6391c0de4c7 h1:YcyjlL1PRr2Q17/I0dPk2JmYS5CDXfcdb2Z3YRioEbw=
+google.golang.org/genproto/googleapis/api v0.0.0-20240826202546-f6391c0de4c7/go.mod h1:OCdP9MfskevB/rbYvHTsXTtKC+3bHWajPdoKgjcYkfo=
+google.golang.org/genproto/googleapis/rpc v0.0.0-20240826202546-f6391c0de4c7 h1:2035KHhUv+EpyB+hWgJnaWKJOdX1E95w2S8Rr4uWKTs=
+google.golang.org/genproto/googleapis/rpc v0.0.0-20240826202546-f6391c0de4c7/go.mod h1:UqMtugtsSgubUsoxbuAoiCXvqvErP7Gf0so0mK9tHxU=
 google.golang.org/protobuf v1.36.12 h1:pJOKDDOyeXErUroCihFAd5LQuwXBSpVnKGrj5o/fwxc=
 google.golang.org/protobuf v1.36.12/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
+gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
+gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
 gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
 gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
 gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

internal/detection/README.md

@@ -0,0 +1,36 @@
+# Declarative detection rules
+
+The built-in pack is embedded from `rules/*.yaml` and compiled once when the
+ingestion worker starts evaluating a job. `LoadPack` can validate an external
+directory for a bounded backtest or a future rule-management API.
+
+Each rule has a stable `id` and semantic `version`. `source` is an allow-list
+for provider/event types; `when.expression` is CEL evaluated with only these
+bindings:
+
+- `event`: immutable event metadata and provider payload
+- `org.config`: non-secret tenant configuration supplied by the caller
+- `org.allowlists`: non-secret tenant allowlists supplied by the caller
+- `now`: RFC3339 timestamp for deterministic evaluation
+
+Finding strings and evidence use a logic-free template subset. Dotted paths,
+`first_nonempty(path, path, ...)`, and
+`external_recipient(path, path, owner_path)` are supported. Templates cannot
+call CEL, access the filesystem, or execute Go code. Unknown YAML fields,
+invalid semantic versions, invalid severities, oversized expressions, and
+duplicate rule IDs are rejected before compilation. A pack has one active
+semantic version per rule ID; replacing a rule version is an explicit pack
+rollout rather than an in-place overlap.
+
+`auto_resolve_when` emits a resolution draft only. Persistence must scope the
+state transition by `organization_id`, `integration_id`, rule ID, and rendered
+dedupe target; the evaluator never mutates storage.
+
+The evaluator exposes version-aware dedupe material for backtests. The current
+worker preserves the existing persisted ID-plus-target hash during migration
+and records the rule version in finding evidence; changing that database key
+requires a separate collision/backfill rollout.
+
+The JSON schema in `rule.schema.json` is a portable authoring contract. The Go
+loader additionally compiles every expression, which catches CEL errors before
+activation.