PII scrubbing: organization-level defaults with project overrides

[unjica]

Roadmap priority

8

Goal

Add organization-level PII scrub defaults that apply across projects in an organization, with clear merge/override rules relative to per-project settings.

This issue is the first concrete slice of the organization defaults model described in #497.

Current state

• Project-scoped PII scrubbing is complete via PII scrubbing for ingest and stored telemetry #470 (deny-keys, session email scrubbing, audit logs, and backfill).
• Teams with many projects still must configure each project separately for shared deny-keys / session-email policy.
• There is no organization-level PII defaults layer yet.
• Broader org defaults (retention, notifications, quotas, alerts, AI) are tracked in Organization settings: defaults layer with project overrides #497 — implement PII only here.

Motivation

Teams with many projects want shared deny-keys / session-email policy without configuring each project. Project overrides should remain possible for exceptions.

Proposed direction

• New org-scoped settings (schema + API under /api/meta/organizations/:orgId/… or equivalent)
• Merge semantics documented and tested (see below)
• Cache invalidation across all projects in the org (or keyed cache that includes org settings)
• Dashboard: Organization settings UI for PII defaults + keep project settings for overrides
• UI clearly indicates inherited vs overridden values (aligned with Organization settings: defaults layer with project overrides #497)
• Audit events on org settings changes (similar to project.pii_scrub.update)
• Permissions: OWNER (or EDITOR+) consistent with other org settings

Merge semantics (PII-specific)

Organization settings act as the defaults layer; project settings remain the override layer (#497).

Document and implement precedence for:

Setting	Proposed merge (confirm in PR)
Deny-keys	Org defaults additive with project deny-keys (union), unless a project explicitly opts out of inheritance
scrubSessionUserEmail (boolean)	Document precedence: project override wins when set; otherwise inherit org default

Organization defaults must not silently remove supported project-level override capabilities.

Out of scope

• Custom regex controls — companion follow-up issue
• Automatic backfill when org defaults change (operators can use the existing CLI)
• Organization defaults for non-PII capabilities (retention, notifications, quotas, alerts, AI) — see Organization settings: defaults layer with project overrides #497
• RBAC redesign / team permissions overhaul

Related

• PII scrubbing for ingest and stored telemetry #470 — Project-scoped PII scrubbing (done)
• Organization settings: defaults layer with project overrides #497 — Organization settings: defaults layer with project overrides (parent model)
• Companion follow-up — custom regex controls (if filed)

Base branch

develop

Suggested acceptance checks

1. Migration + parse/validate helpers ship with safe defaults.
2. Ingest loads merged settings without cross-org leakage.
3. Project overrides behave according to documented merge rules.
4. Dashboard load/save/error states work; inherited vs overridden values are clear.
5. Cache invalidation covers org setting changes.
6. Authorization is enforced consistently for org PII settings.
7. Tests cover merge rules, authz, cache invalidation, and audit events.
8. Docs describe org vs project vs SDK layers for PII scrubbing.

[unjica]

8:Maps to roadmap org PII defaults under Organization Settings (priority 8). Broader org defaults tracked in #497.