Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
e724e0c
docs: establish protected-main product requirements
seonghobae Aug 14, 2026
25d7eac
docs: establish protected-main technical requirements
seonghobae Aug 14, 2026
6f8e6c9
docs: add canonical documentation fitness inventory
seonghobae Aug 14, 2026
7a17e5f
docs: add protected-main traceability map
seonghobae Aug 14, 2026
1cbc9ea
docs: index protected-main architecture decisions
seonghobae Aug 14, 2026
0562330
docs: record active ADR index reconstruction
seonghobae Aug 14, 2026
b708dbb
docs: refresh atomic result application status
seonghobae Aug 14, 2026
65338df
docs: align traceability with active result application
seonghobae Aug 14, 2026
b6ce4d0
docs: refresh runtime provisioning overlay status
seonghobae Aug 14, 2026
7f1dbb4
docs: refresh active runtime-store traceability
seonghobae Aug 14, 2026
fecc977
docs: trace secret encryption policy status
seonghobae Aug 15, 2026
b2ea322
docs: make lifecycle compatibility contract explicit
seonghobae Aug 15, 2026
f0cf1f7
docs: pin tenant lifecycle technical invariants
seonghobae Aug 15, 2026
b9f15c0
docs: verify tenant lifecycle companion authority
seonghobae Aug 15, 2026
2a540ba
docs: define utility-preserving PII controls
seonghobae Aug 15, 2026
6279c63
docs: specify content-fidelity privacy boundary
seonghobae Aug 15, 2026
8c01d37
docs(security): clarify tenant setting trust boundary
seonghobae Aug 15, 2026
d55b21e
docs(security): mirror tenant setting authority invariant
seonghobae Aug 15, 2026
6b36385
docs(traceability): register active validation privacy repair
seonghobae Aug 15, 2026
3d5f79f
docs(traceability): record current PR 202 blocker
seonghobae Aug 16, 2026
3525ab6
docs(security): clarify tenant and result authority
seonghobae Aug 16, 2026
20b5362
docs(privacy): bound result and retention contracts
seonghobae Aug 16, 2026
008aa98
docs(recovery): trace integrated evidence and active executors
seonghobae Aug 16, 2026
e207682
docs(recovery): refresh documentation fitness at protected main
seonghobae Aug 16, 2026
e5901d1
docs(recovery): align product contract with protected main
seonghobae Aug 16, 2026
ad761d3
docs(recovery): align technical contract with protected main
seonghobae Aug 16, 2026
458a105
docs(recovery): refresh ADR index authority and recovery boundary
seonghobae Aug 16, 2026
89f4e52
docs(recovery): mark seek-safe restore successor
seonghobae Aug 16, 2026
eafbfa9
docs(recovery): trace seek-safe restore successor
seonghobae Aug 16, 2026
229f8d3
Merge branch 'main' into docs/canonical-documentation-authority
opencode-agent[bot] Aug 16, 2026
c27075a
docs(recovery): rebase restore overlay onto merged main
cursoragent Aug 16, 2026
85949a2
docs(recovery): name overlay #214 in the register
cursoragent Aug 16, 2026
8227e82
docs: refresh canonical active-overlay truth
seonghobae Aug 16, 2026
c76ca29
docs(recovery): drop transient Draft labels from canonical status
cursoragent Aug 16, 2026
ad39b4d
docs: add threat model and package-owned schema ERD
cursoragent Aug 16, 2026
65be01d
docs: add data-governance and component UML overlays
cursoragent Aug 16, 2026
93b76d2
docs(recovery): name the live canonical overlay without Draft instruc…
cursoragent Aug 16, 2026
3a8739b
chore(pr229): reconcile canonical docs onto current protected main
seonghobae Sep 10, 2026
af038e5
chore(pr229): stack canonical docs on recovery coverage prerequisite
seonghobae Sep 10, 2026
5743a34
test(docs): reject stale merged-capability status
seonghobae Sep 10, 2026
d8c513f
docs(product): align shipped recovery and secret policy truth
seonghobae Sep 10, 2026
3490a81
docs(technical): repair merged recovery capability status
seonghobae Sep 10, 2026
430c0c6
docs: repair canonical documentation fitness truth
seonghobae Sep 10, 2026
d6d09aa
docs(trace): remove stale active recovery authorities
seonghobae Sep 10, 2026
6df9658
docs(adr): align index with integrated restore decision
seonghobae Sep 10, 2026
48e97d1
docs(security): bound optional Fernet compatibility risk
seonghobae Sep 10, 2026
47b2849
docs(data): distinguish compatibility obfuscation from encryption
seonghobae Sep 10, 2026
bd468d2
test(docs): require merged restore-target identity truth
seonghobae Sep 10, 2026
bed5ba2
docs(product): record merged restore-target identity boundary
seonghobae Sep 10, 2026
0587d9d
docs(technical): record merged cluster identity verifier
seonghobae Sep 10, 2026
29c400a
docs: promote merged restore-target identity seam precisely
seonghobae Sep 10, 2026
7400c29
docs(trace): promote merged restore-target cluster verifier
seonghobae Sep 10, 2026
39a2902
docs(adr): index integrated restore-target decision
seonghobae Sep 10, 2026
314c8b5
docs(security): model bounded restore-target identity trust
seonghobae Sep 10, 2026
7f3acbd
test(docs): align historical lineage and capability assertions
seonghobae Sep 10, 2026
ced97af
docs: preserve owned overlay path fitness contracts
seonghobae Sep 10, 2026
c7b32be
docs(security): restore tested asset identifiers and standalone action
seonghobae Sep 10, 2026
792c24e
docs(recovery): record PITR target observation boundary
seonghobae Sep 11, 2026
221fd52
docs(traceability): track recovery target observation overlay
seonghobae Sep 11, 2026
045e42b
test(docs): pin PITR observation documentation boundary
seonghobae Sep 11, 2026
bf6dd65
fix(docs): align PITR observation traceability contract
seonghobae Sep 11, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
80 changes: 80 additions & 0 deletions docs/DATA_GOVERNANCE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
# Data governance

## Authority

This contract maps protected-main data classes, owners, tenant authority,
retention, deletion, and privacy boundaries. SQL, package code, and the
canonical PRD/TRD remain stronger authority. It is evidence readiness, not a
certification and not a claim that a deployment has a complete records program.

## What to do next

1. Decide which host identity is allowed to select `tenant_scope` before any
package call.
2. Do not mask, tokenize away, or truncate authorized business payloads inside
this package. If your policy requires transformation, do it in an explicit
host boundary with provenance and acceptance tests.
3. Treat Fernet as an optional host/deployment policy on protected main.
`SecretStore(require_encryption=False)` permits base64-obfuscated
compatibility rows with `com_secrets.is_encrypted = FALSE`; that mode is not
a mandatory encryption-at-rest guarantee. Historical-row migration, key
rotation/recovery, and external key custody remain separate responsibilities.
4. Own backup copies, replica retention, log/telemetry retention, and
destructive deletion yourself. The package will not invent a general purge.
5. Use `standalone` for a single-tenant operator. Do not reuse that scope as an
anonymous public bucket.

## Data classes

| Class | Principal objects | Owner | Package duty |
| --- | --- | --- | --- |
| Authorized business payloads | `llm_requests` prompts/results, `llm_batch_file_payloads`, `llm_jsonl_lines` | Embedding host / business process | Persist and replay exactly. Do not mask. |
| Durable lifecycle projection | `llm_remote_batch_jobs` | Host-selected `tenant_scope` plus package recorder | Tenant-qualify identity, bind `set_config`, force RLS. |
| Result checkpoints | `llm_result_stream_checkpoints` | Host-selected consumer name plus tenant | Store prefix evidence only. |
| Standalone configuration | `com_config` | Operator | Key/value settings. Not tenant authorization. |
| Standalone secrets | `com_secrets` | Operator / secret-manager host | Optional Fernet or explicit compatibility mode. Compatibility rows are base64-obfuscated, not an encryption-at-rest claim. |
| Provider credentials | Host credential provider | Deployment | Resolve after tenant validation. Never tenant-keyed by this package. |
| Recovery evidence | Receipts, artifact hashes, schema hashes | Operator | Content-free identity. Not restorability. |
| Operational diagnostics | Errors, logs, readiness, telemetry | Package | Omit payloads, DSNs, credentials, and dynamic exception text. |

## Tenant authority

`tenant_scope` is selected only by a trusted authenticated/authorized host
boundary. Provider metadata, remote identifiers, request bodies, model output,
endpoint aliases, and transport headers are never tenant authorities. The
embedding host owns the identity-to-tenant map.

## Retention and deletion

Protected main does not define a universal business-data retention duration.
The embedding host owns purpose, retention period, deletion authorization, and
evidence that the policy ran. The deployment owner separately owns PostgreSQL
backup/replica, WAL, and infrastructure log retention. Provider-side retention
remains a provider/account policy unless a reviewed adapter implements it.

The package will not silently delete unknown operator objects, rewrite history,
or `CASCADE` through unrelated schemas as a recovery shortcut.

## Privacy without paralysis

ISO/IEC 29100 treats purpose specification and data minimization as
organization policy, not as an excuse to destroy the meaning of a processing
record (ISO/IEC, 2024). NIST SP 800-53 Revision 5 likewise places confidentiality
controls at authorization, access, transmission, and audit boundaries (Joint
Task Force, 2020). This package therefore:

- preserves authorized business payloads;
- redacts operational surfaces;
- fails closed on untrusted provider and path input;
- refuses to treat RLS or diagnostic redaction as proof that persisted
business content was masked.

## References

International Organization for Standardization. (2024). *Information
technology — Security techniques — Privacy framework* (ISO/IEC 29100:2024).
https://www.iso.org/standard/85938.html

Joint Task Force. (2020). *Security and privacy controls for information systems
and organizations* (NIST Special Publication 800-53, Revision 5). National
Institute of Standards and Technology. https://doi.org/10.6028/NIST.SP.800-53r5
59 changes: 59 additions & 0 deletions docs/DOCUMENTATION_FITNESS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
# Documentation Fitness

## Authority and status model

This inventory evaluates canonical documentation against live protected-default-branch behavior. It deliberately does not freeze an exact protected SHA: exact heads and run IDs belong in PR/review evidence, while durable documentation records capability contracts. Status vocabulary is **IMPLEMENTED-ON-PROTECTED-MAIN**, **ACTIVE-PR**, **PARTIAL**, **PLANNED**, and **SUPERSEDED**.

A document is fit only when it agrees with protected code/schema/tests, preserves non-guarantees, keeps branch evidence out of shipped claims, and gives operators/reviewers enough information to use or reject the behavior safely.

Protected main contains the tenant lifecycle/RLS contract, bounded reconciliation, tenant-qualified transient session single-flight integrated through #191, bounded recovery-evidence primitives integrated through #205/#206/#207, bounded direct logical restore integrated through #212, and bounded restore-target name+cluster-identity verification integrated through merged #228. These are narrower than a complete worker or recovery product.

## Current fitness matrix

| Documentation surface | Status | Fitness assessment | Required next action |
| --- | --- | --- | --- |
| `README.md` | IMPLEMENTED-ON-PROTECTED-MAIN | Public entry point for standalone/embedded operation. | Change through its live owner when protected behavior changes. |
| `ARCHITECTURE.md` | IMPLEMENTED-ON-PROTECTED-MAIN | Root architecture is separately owned by the active root-documentation lane. | Keep this PR out of that path. |
| `docs/product/PRD.md` | ACTIVE-PR | Canonical product contract records merged #191, #212, and #228 only at their bounded protected scope. | Revalidate from protected source after every integration. |
| `docs/product/TRD.md` | ACTIVE-PR | Separates transient session exclusion from durable leasing; restore execution from backup/application/PITR; and cluster-identity comparison from connection provenance/authorization. | Keep source/test authority stronger than prose. |
| ADR set | IMPLEMENTED-ON-PROTECTED-MAIN / record-local | ADR 0016 governs custom-format restore seek semantics; ADR 0022 governs restore-target name+cluster identity separation. Other ADRs retain their own status. | Preserve record-local status and collision-free identifiers. |
| `docs/adr/README.md` | ACTIVE-PR | Navigation/status index without exact-head authority. | Keep synchronized with protected ADR files. |
| Tenant lifecycle operator material | IMPLEMENTED-ON-PROTECTED-MAIN | Trusted tenant selection, standalone compatibility, forced RLS, direct-SQL limits, migration, and rollback are documented. | Do not duplicate into competing operator guides. |
| PostgreSQL recovery evidence | IMPLEMENTED-ON-PROTECTED-MAIN / end-to-end PARTIAL | Receipt/artifact/schema evidence is bounded content-free identity/integrity evidence. | Do not infer restorability, provenance, PITR, or RPO/RTO. |
| PostgreSQL logical backup | ACTIVE-PR | #208 remains a `pg_dump` candidate. | Keep unshipped until normal integration. |
| PostgreSQL logical restore | IMPLEMENTED-ON-PROTECTED-MAIN / end-to-end PARTIAL | The logical restore executor is protected-main behavior through merged #212; #209 is historical EOF-defect evidence. | Preserve source trust, environment, transaction, metadata, target, and application-readiness boundaries. |
| PostgreSQL restore-target cluster identity verification | IMPLEMENTED-ON-PROTECTED-MAIN / end-to-end PARTIAL | Merged #228 supplies exact service-name plus caller-owned `system_identifier` separation. Same-cluster aliases fail closed. The package does not open/authenticate the connections, execute restore, or prove application/PITR/RPO-RTO readiness. Closed #225 is predecessor lineage only. | Treat the verifier as a bounded precondition, not end-to-end restore authorization. |
| Effective PITR target configuration observation | ACTIVE-PR | #299 reads exactly eight PostgreSQL recovery-target settings plus `pg_is_in_recovery()` from a caller-owned isolated target and compares them with reviewed target authority. The functions remain module-scoped rather than root-package exports. | Keep fixed-query, timeout-ownership, public-surface, README/CHANGELOG/ADR/doctoring work with their live owners; do not promote branch evidence to shipped PITR proof. |
| Recovery evidence binding / live reinspection | ACTIVE-PR | Candidate composition/reinspection is not provenance or restore proof. | Keep branch evidence distinct from shipped truth. |
| Post-restore catalog/application acceptance | ACTIVE-PR | #296 is an application-readiness candidate. | Require protected integration before shipped claims. |
| Permanent live PostgreSQL integration acceptance | ACTIVE-PR | #341 owns the branch-level full integration-marker lane and currently tests #296 as a child. | Preserve the lane through normal integration; no mocks/deselection substitution. |
| Physical/WAL/PITR recovery | ACTIVE-PR / PARTIAL | Intent/evidence does not prove replay/promotion or achieved objectives. | Require deployment-specific execution and measurement. |
| Existing-volume legacy extension retirement | ACTIVE-PR | Separate migration/operator work. | Do not conflate with tenant lifecycle. |
| Durable reconciliation discovery | ACTIVE-PR | Discovery remains unshipped. | Keep tenant-qualified/bounded/deterministic authority. |
| Tenant-qualified reconciliation single-flight | IMPLEMENTED-ON-PROTECTED-MAIN | #191 is transient PostgreSQL session advisory locking only. | Never call it a scheduler, durable lease, result-app transaction, terminal-retirement authority, or distributed exactly-once mechanism. |
| Atomic durable result application | ACTIVE-PR / end-to-end PARTIAL | Checkpoint/stream primitives do not prove complete result application. | Keep external-effect limits explicit. |
| Runtime config/schema provisioning and secret policy | ACTIVE-PR / protected compatibility baseline | Protected main permits optional Fernet and `is_encrypted = FALSE` compatibility rows; #210 is stricter active work. | Do not claim mandatory encryption, historical-row migration, rotation/recovery, or external custody. |
| Canonical traceability | ACTIVE-PR | This overlay is the current canonical documentation landing vehicle; #226 and superseded #214 are historical predecessors. | Use stable implementation/test/doc authorities, not exact heads. |
| `docs/THREAT_MODEL.md` | ACTIVE-PR | Assets, boundaries, mitigations, residual risk, and NIST evidence are documented without certification claims. | Keep residual risk synchronized with protected authority. |
| `docs/DATA_GOVERNANCE.md` | ACTIVE-PR | Data classes, owners, retention/deletion, content fidelity, and optional Fernet compatibility are explicit. | Do not turn evidence readiness into certification. |
| `docs/uml/component-and-sequence.md` | ACTIVE-PR | Standalone/embedded and tenant-validation views exist. | Keep branch-only components off shipped diagrams. |
| `docs/erd/package-owned-schema.md` | ACTIVE-PR | Packaged schema and migration-owned checkpoint identity are mapped. | SQL remains stronger authority. |
| Release governance | PARTIAL | Release evidence exists; immutable publication requires the exact accepted protected head. | Tie version, CHANGELOG, package, SBOM, provenance, rollback, tag, and publication verification together through release ownership. |

## Non-negotiable documentation invariants

- Protected-main behavior is shipped authority; active PRs and historical branches are not.
- Exact SHAs, generated merge commits, run IDs, and queue state stay in PR/review evidence.
- Standalone and modular embedding remain co-equal boundaries.
- `tenant_scope` comes from a trusted authenticated/authorized host; RLS is defense in depth, not authentication.
- #191 proves transient tenant-qualified session single-flight, not durable leasing or exactly-once.
- #212 proves bounded direct logical restore with corrected custom-format seek semantics, not backup, application readiness, PITR, or RPO/RTO.
- #228 proves only that supplied exact service names and caller-owned PostgreSQL `system_identifier` values differ. It does not authenticate the connections or collector, execute restore, or prove post-restore application readiness.
- #299 is an ACTIVE-PR fixed-query observation of effective recovery-target settings on a caller-owned isolated target. Its functions are module-scoped, it owns no connection timeout, and it does not prove WAL completeness, target attainment, promotion, application readiness, PITR success, or achieved RPO/RTO.
- Optional Fernet plus explicit compatibility mode is protected behavior; mandatory encryption/migration/rotation/custody is not inferred.
- Recovery evidence and command success are not equivalent to recovery success.
- SOC 2/CSAP/security/privacy material remains evidence readiness absent external certification.

## Fitness gate

Before changing a canonical surface, refetch protected main, open PRs, affected source/schema/tests, current ADRs, and adjacent writers. Repair the earliest stale authority boundary without widening ownership. After a merge, refresh status only from the resulting protected tree; after supersession, retain predecessor context only where it explains a live constraint. Every changed documentation head must reacquire then-required exact-head quality/security/release evidence and qualifying review.
Loading