diff --git a/docs/adr/0016-governed-selection-outcome-monitoring-plan.md b/docs/adr/0016-governed-selection-outcome-monitoring-plan.md new file mode 100644 index 000000000..48dfa673b --- /dev/null +++ b/docs/adr/0016-governed-selection-outcome-monitoring-plan.md @@ -0,0 +1,48 @@ +# ADR 0016: Governed selection-outcome monitoring plan + +- **Status:** Proposed — active PR only +- **Decision scope:** Selection validity / workforce intelligence governance + +## Context + +Orgmetra already owns Job-scoped selection and post-hire evidence boundaries, but protected `develop` does not define a buyer-facing contract for planning recurring selection-outcome monitoring without copying candidate-level protected-attribute values or turning a screening heuristic into an automated legal or employment decision. Different opaque requester/reviewer references also do not prove that the authoritative actor boundary resolves them to different accountable people, and valid UUID-backed references alone do not prove that all referenced evidence belongs to the packet tenant. Packet-owned UUIDv1 trust references additionally embed timestamp/node-derived correlation metadata. The authoritative tenant identifier is different: it is issued by Orgmetra core, so this leaf package must accept the canonical non-sentinel operational UUID contract owned by that boundary rather than silently imposing a second version policy. + +The EEOC's common interpretation of the Uniform Guidelines directs users to examine the total selection process first for each job, describes the four-fifths rule as a rule of thumb rather than a legal definition, and notes that small samples, statistical significance, practical significance, and other evidence can matter. ISO 30405:2023 also treats reviewing and learning as part of recruitment practice. SIOP's fifth-edition Principles provide the professional validation framework for personnel selection procedures. + +## Decision + +Orgmetra will expose a transport-neutral `SelectionOutcomeMonitoringPlan` that binds: + +- one canonical non-sentinel operational tenant under the authoritative Orgmetra core contract and one authoritative Job; +- one total selection-process reference; +- exact aggregate population and selection-outcome snapshot references and SHA-256 digests; +- exact protected-attribute handling, small-sample interpretation, and statistical-analysis plan references and digests; +- an accountable requester reference and an accountable reviewer reference; +- fixed purpose and reviewed reason metadata plus a bounded positive `evidence_version` that is part of canonical evidence; +- an explicit monitoring business-date window and evidence-generation instant. + +`tenant_record_id` must be canonical and non-sentinel under Orgmetra's authoritative operational UUID contract. The package does not reinterpret its UUID version because tenant identity generation and migration policy belong to the authoritative HRIS boundary. Packet-owned namespaced trust-bearing references separately require canonical non-sentinel UUIDv4 plus their expected prefix. UUIDv1 and other non-v4 suffixes fail closed for those references. Human-readable, value-bearing, sentinel, and noncanonical reference suffixes are also rejected so labels, policy values, protected-attribute concepts, or actor names cannot be carried through fields represented as opaque identifiers. `evidence_version` must be a true integer from 1 through 2147483647; changing it changes canonical JSON and the packet SHA-256, so revisions to actor/purpose/reason-bound evidence cannot silently collide. + +UUID syntax is not tenant authority. Before review, the host must re-resolve **every packet reference** within the exact `tenant_record_id` through the relevant authoritative boundary and reject review use if any reference belongs to another tenant or cannot be authoritatively resolved. The packet also rejects identical requester/reviewer references as an early syntactic guard; after tenant-scoped resolution, the host must prove that the two references resolve to distinct actor identities. Reference inequality alone is not separation-of-duties evidence. + +The contract is aggregate-only and carries no candidate identity, protected-attribute value, individual assessment score, individual employment decision, or free-form model output. It fixes `analysis_scope` to `total_selection_process_by_job`, `decision_authority` to `human_review_only`, and state to `requires_human_review`. It does not calculate selection rates, mechanically apply the four-fifths heuristic, test statistical significance, infer discrimination, or authorize a process change. + +Each live plan identity receives one process-local construction seal over its exact canonical bytes. Seal registration is one-shot: a repeated `__post_init__()` call cannot overwrite the original seal, including after low-level mutation to another syntactically valid value. Canonical export first revalidates every trust-bearing text field and each date, datetime, integer, and boolean evidence field against its exact built-in runtime type before invoking field rendering or serialization. That keeps behavior-bearing post-issuance subtypes from retaining authority merely because they serialize to the same bytes. Export then compares the resulting canonical payload with the original construction evidence instead of permitting reinitialization or representation-preserving aliases to renew trust. This runtime mechanism is defense-in-depth only and does not replace durable immutable audit/outbox evidence, persistence uniqueness, or cross-process authorization. + +Any later analytics or persistence boundary must independently enforce purpose-bound authorization, authoritative tenant-scoped reference and actor resolution, minimum-necessary protected-attribute access, small-sample controls, provenance, immutable audit evidence, and accountable human interpretation. Results are evidence for review, not an automated high-impact employment decision or certification/legal conclusion. + +## Consequences + +- Buyers obtain a deterministic, explicitly versioned governance envelope for recurring selection monitoring without creating a second psychometrics/statistics engine inside Orgmetra. +- The total-process-by-Job scope is explicit before any future component drill-down. +- Privacy risk is reduced because individual protected-attribute values and candidate records remain outside the plan envelope and packet-owned trust references reject UUIDv1 timestamp/node metadata and value-bearing suffixes without making the leaf package incompatible with authoritative Orgmetra tenant UUIDs. +- Cross-tenant evidence mixing is fail-closed at the host review boundary because every opaque reference must be re-resolved in the exact packet tenant. +- Requester/reviewer separation is proven from authoritative resolved actor identities rather than inferred from different opaque strings. +- A valid-value low-level rewrite cannot be legitimized by re-running dataclass initialization because process-local seal registration is single-use for the live identity. +- A post-issuance runtime subtype cannot preserve evidence authority solely by keeping canonical JSON bytes unchanged; exact built-in runtime types are re-established before canonical export invokes field behavior. +- The four-fifths rule cannot be represented as an automatic pass/fail legal rule by this contract; interpretation remains with authorized analysts and accountable humans. +- Psychometric/statistical production compute remains owned by the appropriate Psychometrics Commons / fast-mlsirm / TEPP contract when those kernels are needed. + +## References + +See `docs/doctoring/selection-outcome-monitoring-references.md`. diff --git a/docs/doctoring/selection-outcome-monitoring-references.md b/docs/doctoring/selection-outcome-monitoring-references.md new file mode 100644 index 000000000..c86098cee --- /dev/null +++ b/docs/doctoring/selection-outcome-monitoring-references.md @@ -0,0 +1,17 @@ +# Selection-outcome monitoring references + +These references support ADR 0016 and the bounded selection-monitoring contract. They do not convert Orgmetra into a legal-advice, certification, or automated adverse-impact decision service. + +## APA 7 references + +Equal Employment Opportunity Commission, Office of Personnel Management, Department of Justice, Department of Labor, & Department of the Treasury. (1979, March 2). *Questions and answers to clarify and provide a common interpretation of the Uniform Guidelines on Employee Selection Procedures*. U.S. Equal Employment Opportunity Commission. https://www.eeoc.gov/laws/guidance/questions-and-answers-clarify-and-provide-common-interpretation-uniform-guidelines + +International Organization for Standardization. (2023). *ISO 30405:2023 Human resource management—Guidelines on recruitment* (2nd ed.). https://www.iso.org/standard/79488.html + +Society for Industrial and Organizational Psychology. (2018). Principles for the validation and use of personnel selection procedures (5th ed.). *Industrial and Organizational Psychology, 11*(S1), 1–97. https://doi.org/10.1017/iop.2018.195 + +## Applied evidence boundary + +The EEOC source says adverse impact is examined first for the overall selection process for each job and describes the four-fifths/eighty-percent rule as a practical rule of thumb rather than a legal definition. It also explains that small samples and statistical/practical significance can change interpretation. Consequently, the Orgmetra contract binds a Job-scoped total-process monitoring plan, a separate small-sample policy, and a separate statistical plan but does not itself calculate or adjudicate adverse impact. + +ISO 30405:2023 is the current published second edition and includes reviewing and learning among recruitment practices. SIOP's fifth-edition Principles are used as the professional selection-validation frame; Orgmetra does not duplicate its psychometric methods in this package. diff --git a/docs/traceability/selection-outcome-monitoring.md b/docs/traceability/selection-outcome-monitoring.md new file mode 100644 index 000000000..48dc17fef --- /dev/null +++ b/docs/traceability/selection-outcome-monitoring.md @@ -0,0 +1,33 @@ +# Selection-outcome monitoring traceability + +## Status + +**Active PR / proposed capability.** This file does not describe protected-`develop` behavior until the owning PR is integrated. The protected-parent reconciliation snapshot is `develop@eb9757f8649aaad026a9865508d9aad50c1a7a4f`; merge readiness still requires a fresh protected-head read and exact-head acceptance. + +## Buyer need → contract evidence + +| Buyer / governance need | Owned contract evidence | Explicit non-claim | +|---|---|---| +| Monitor the correct hiring/promotion process | Canonical non-sentinel `tenant_record_id` under the Orgmetra core operational-UUID contract, exact UUIDv4-backed `job_profile_reference` and `selection_process_reference`; fixed `analysis_scope=total_selection_process_by_job`; immutable next action requires every packet reference to be re-resolved within exact `tenant_record_id` | UUID syntax alone is not tenant authority or component-level causality evidence | +| Reproduce the monitored population and outcomes | Exact UUIDv4-backed aggregate population/outcome snapshot references plus independent SHA-256 digests | No candidate-level record or protected-attribute value in the packet | +| Preserve privacy and interpretation rules | Exact UUIDv4-backed protected-attribute handling and small-sample policy references/digests | No blanket authorization to expose protected-attribute data | +| Prevent semantic/value/correlation smuggling through packet-owned references without duplicating tenant identity policy | `tenant_record_id` is canonical/non-sentinel under the authoritative core contract; every packet-owned governed reference requires canonical non-sentinel UUIDv4 plus its expected prefix; digest and governance text evidence require exact built-in strings before validation | UUID syntax does not prove source truth, tenant membership, or authorization | +| Prevent cross-tenant evidence mixing | `test_actor_separation.py` requires the governed next action to re-resolve every packet reference within `tenant_record_id` before actor separation, Job scope verification, or accountable review | The packet does not itself query authoritative stores | +| Bind the analysis method before interpretation | Exact UUIDv4-backed statistical-plan reference/digest | No statistics are calculated by this package | +| Version actor/purpose/reason evidence explicitly | `evidence_version` is a true positive integer through signed-int32 max and participates in canonical JSON/SHA-256 | `test_evidence_version.py` proves presence, digest separation, bounds, and `dataclasses.replace(...)` revalidation | +| Prove accountable requester/reviewer separation | Different opaque actor references as a syntactic guard plus tenant-scoped authoritative resolution requiring distinct resolved actor identities | Reference inequality alone is not identity or separation-of-duties evidence | +| Prevent automated high-impact action | Exact boolean human confirmation, `human_review_only`, `requires_human_review`, governed next action | No automated employment-process change or legal conclusion | +| Preserve replayable audit correlation without caller-owned timezone behavior | `generated_at` is resolved once to a built-in UTC instant at issuance; future instants, missing/raising offsets and normalization overflow fail closed; canonical JSON and SHA-256 reuse only the detached UTC value | The packet timestamp proves evidence chronology/correlation, not source truth or scientific/legal validity | +| Prevent valid-value evidence rewrites after issuance | A process-local HMAC seal is stored outside packet-writable slots over the exact construction-time canonical JSON; seal registration is single-use per live identity, so repeated `__post_init__()` cannot renew trust; export verifies the current snapshot against the original seal and fails closed if evidence changed or issuance state is unavailable | The process-local seal is defense-in-depth only and is not durable cross-process authorization, persistence uniqueness, or immutable audit/outbox evidence | +| Prevent representation-preserving executable aliases after issuance | Before canonical rendering, every serialized trust-bearing text field and date/datetime/int/bool evidence field is rechecked against its exact built-in runtime type. Only after that non-executable boundary passes may date/timestamp rendering, JSON serialization and construction-seal verification execute. | Runtime-type integrity does not replace authoritative tenant/reference re-resolution, durable audit/outbox, or scientific/legal validity evidence | +| Keep package-quality evidence current under the protected workflow-ownership model | Canonical `Foundation CI` invokes `tests/test_foundation_ci_dependency_hygiene.sh`, which discovers and executes `tests/test_foundation_ci_*_artifact.sh` contracts; Selection Monitoring is owned by `tests/test_foundation_ci_selection_monitoring_artifact.sh`. The isolated hash-locked package contract runs the package pytest configuration, whose 100% statement and branch coverage thresholds remain authoritative. `test_quality_workflow_trigger.py` fails if the retired leaf workflow reappears or this canonical delegation disappears. | There is no package-local Selection Monitoring workflow and no transfer of historical GREEN across a successor head | + +## Executable evidence + +`packages/selection-monitoring/tests/test_plan.py` exercises direct-constructor and builder validation, operational tenant identity, UUID-backed reference namespaces, SHA-256 digests, requester/reviewer syntactic separation, monitoring-window boundaries, governance codes, timezone handling, fractional-second evidence identity, immutable review/authority state, aggregate-only enforcement, canonical JSON, and deterministic packet hashing. `packages/selection-monitoring/tests/test_temporal_evidence_integrity.py` proves caller-defined datetime subclasses are rejected, mutable timezone providers are detached at issuance, future generation times and missing/raising offsets fail closed, UTC-normalization overflow is normalized to validation failure, and post-construction non-UTC reinjection is rejected before evidence export. `packages/selection-monitoring/tests/test_issuance_integrity.py` proves a low-level valid-value rewrite after issuance cannot emit a second canonical truth, repeated `__post_init__()` cannot overwrite the original construction seal, and missing process-local issuance evidence fails closed. `packages/selection-monitoring/tests/test_post_issuance_runtime_type_integrity.py` proves a same-value `str` subtype cannot preserve issuance authority and an executable `date` subtype is rejected before its overridden `isoformat()` can run. `packages/selection-monitoring/tests/test_fixed_governance_runtime_integrity.py` proves fixed governance fields reject hostile runtime string subclasses before equality-based policy checks. `packages/selection-monitoring/tests/test_string_runtime_evidence_integrity.py` proves tenant, reference, purpose, reason, and SHA-256 digest evidence reject caller-defined string subclasses before canonical evidence binding. `packages/selection-monitoring/tests/test_actor_separation.py` requires the immutable next action to re-resolve every packet reference in the exact tenant before Job-scope/accountable-review use, and separately requires requester/reviewer resolution through the authoritative tenant-scoped actor boundary with distinct resolved identities. `packages/selection-monitoring/tests/test_reference_privacy.py` is the RED→GREEN privacy/interoperability contract for accepting the authoritative core UUIDv7 tenant form while rejecting human-readable/value-bearing, sentinel, noncanonical, and non-v4 packet-owned opaque-reference suffixes through both public construction and replacement paths. `packages/selection-monitoring/tests/test_evidence_version.py` requires explicit bounded evidence revision identity in canonical evidence and proves that version changes alter the packet hash. `packages/selection-monitoring/tests/test_quality_workflow_trigger.py` proves the retired package-local workflow stays absent and canonical Foundation ownership continues to discover and execute the package contract without another edit to the shared Foundation workflow. + +The package contract is reached only through canonical `.github/workflows/foundation-ci.yml`. `tests/test_foundation_ci_dependency_hygiene.sh` discovers delegated artifact contracts by the `tests/test_foundation_ci_*_artifact.sh` naming contract, so later packages can add an isolated artifact proof without competing edits to the shared dispatcher. The Selection Monitoring contract creates an isolated virtual environment, installs the repository's hash-locked reviewed test toolchain, compiles the package and tests, runs the selection-monitoring package under its own pytest configuration, and removes the temporary environment on exit. This preserves the existing 100% owned statement/branch threshold without restoring `.github/workflows/selection-monitoring-quality.yml`. Organization-required central review/security workflows remain separate required controls. + +## Ownership boundary + +This slice writes only Orgmetra and introduces no database migration or cross-service SQL. Future statistical computation must use the appropriate published psychometric/statistical service contract rather than duplicating foreign kernels, and future access to protected-attribute data must remain purpose-bound and minimum-necessary. Tenant UUID generation/privacy policy and authoritative tenant-scoped reference/actor resolution remain at the host/core boundary; this packet fails closed by requiring that proof before human review use. The process-local issuance registry is deliberately not a distributed attestation store: durable uniqueness, authorization, retention, and immutable audit/outbox remain responsibilities of authoritative Orgmetra persistence/host boundaries. diff --git a/packages/selection-monitoring/CHANGELOG.md b/packages/selection-monitoring/CHANGELOG.md new file mode 100644 index 000000000..323024166 --- /dev/null +++ b/packages/selection-monitoring/CHANGELOG.md @@ -0,0 +1,15 @@ +# Changelog + +All notable package changes are recorded here. + +## Unreleased + +- Add a governed, aggregate-only `SelectionOutcomeMonitoringPlan` that binds one Job-scoped total selection process to exact aggregate population/outcome snapshots, protected-attribute handling, small-sample interpretation, statistical-plan provenance, a distinct accountable reviewer, and explicit human review without carrying candidate-level values or making an adverse-impact/legal determination. +- Follow Orgmetra's authoritative canonical non-sentinel operational UUID contract for `tenant_record_id`, while every packet-owned namespaced trust-bearing reference remains canonical non-sentinel UUIDv4 and rejects UUIDv1/non-v4, human-readable, value-bearing, sentinel, and noncanonical suffixes through construction and replacement paths. +- Require every packet reference to be re-resolved within the exact tenant through its authoritative boundary before actor separation, Job-scope verification, or accountable review, preventing cross-tenant evidence mixing behind valid opaque UUIDs. +- Bind a true positive `evidence_version` (1..2147483647) into canonical JSON and SHA-256 evidence so revisions to high-impact monitoring evidence cannot silently collide. +- Freeze `generated_at` to a detached built-in UTC instant at issuance, reject future generation times, normalize caller timezone-provider failures to fail-closed validation errors, and prevent later mutable `tzinfo` behavior from rewriting canonical monitoring evidence. +- Bind each live issued monitoring plan to its exact construction-time canonical bytes with a process-local HMAC seal stored outside packet-writable slots; seal registration is single-use per live identity, so a low-level valid-value rewrite followed by repeated `__post_init__()` cannot renew trust. Canonical export fails closed if evidence is rewritten, seal renewal is attempted, or process-local issuance evidence is unavailable. This is defense-in-depth only: durable uniqueness, authorization, and immutable audit/outbox remain authoritative host/persistence responsibilities. +- Revalidate every canonical-export trust field against its exact built-in runtime type before date/timestamp rendering, JSON serialization, or HMAC comparison, so a post-issuance behavior-bearing subtype cannot retain authority merely by preserving the same canonical bytes. +- Require SHA-256 digest evidence to be exact built-in strings before regex validation and canonical binding, matching the package's strict runtime-type policy for other trust-bearing text and rejecting caller-defined `str` subclasses. +- Consolidate selection-monitoring quality into canonical Foundation CI: the retired package-local workflow stays absent, Foundation delegates to an isolated hash-locked package contract, and executable regression keeps 100% statement/branch coverage plus workflow non-resurrection under current protected ownership. diff --git a/packages/selection-monitoring/README.md b/packages/selection-monitoring/README.md new file mode 100644 index 000000000..b2dfe1fb9 --- /dev/null +++ b/packages/selection-monitoring/README.md @@ -0,0 +1,56 @@ +# Orgmetra Selection Monitoring + +`orgmetra-selection-monitoring` defines a governed, aggregate-only evidence packet for planning post-selection outcome monitoring. It is intentionally not a statistics engine, legal decision engine, or candidate-level evidence store. + +## What the contract binds + +A `SelectionOutcomeMonitoringPlan` ties one tenant and authoritative Job to the total selection process being monitored, an aggregate population snapshot, an aggregate selection-outcome snapshot, the protected-attribute handling policy, small-sample interpretation policy, statistical analysis plan, accountable requester and reviewer references, an explicit monitoring window, a system-recorded generation instant, and a bounded positive `evidence_version`. + +`tenant_record_id` follows Orgmetra's authoritative canonical non-sentinel operational UUID contract instead of imposing a second UUID-version policy at this leaf package. Packet-owned trust-bearing artifacts remain canonical non-sentinel UUIDv4 identities; namespaced artifact references additionally require their expected namespace. UUIDv1 and other non-v4 suffixes are rejected for those references so timestamp/node-derived or otherwise nonconforming identifiers cannot masquerade as the package's opaque trust-reference format. Human-readable, value-bearing, sentinel, and noncanonical reference suffixes are also rejected so Job labels, policy values, protected-attribute concepts, actor names, or other sensitive semantics cannot be smuggled through a field described as opaque. Content-bearing evidence adds an independent SHA-256 digest where integrity evidence is required. `reason_code` is closed to the reviewed non-sensitive `quarterly_selection_governance` value for this initial contract, rather than accepting arbitrary lower-snake-case text. `evidence_version` must be a true integer from 1 through 2147483647 and participates in canonical JSON and SHA-256 evidence, so revisions to the actor/purpose/reason-bound monitoring evidence cannot silently collide. Canonical JSON and a packet digest support immutable audit correlation without copying candidate identities, protected-attribute values, assessment scores, individual decisions, or free-form model output. + +`generated_at` is issuance-time evidence rather than a caller-controlled timezone object retained for later execution. Construction requires an exact built-in `datetime`, resolves any concrete `tzinfo` offset once, converts the result to a built-in UTC `datetime`, rejects future instants, and stores only that detached UTC instant. Later canonical export never invokes the caller's original timezone provider, so a mutable or stateful `tzinfo` cannot rewrite already-issued evidence. Provider exceptions, missing concrete offsets, and UTC-normalization overflow fail closed as `ValueError`; low-level reinjection of a non-UTC timestamp also fails before evidence emission. + +A frozen dataclass is not by itself issuance evidence because low-level Python mutation can still rewrite otherwise valid values. Each live issued plan is therefore bound to its exact construction-time canonical JSON by a process-local HMAC seal stored outside packet-writable slots. Seal registration is single-use per live object identity: explicitly re-running `__post_init__()` cannot replace an existing seal after a low-level valid-value rewrite. Before canonical rendering, export rechecks every trust-bearing text field and every date, datetime, integer, and boolean evidence field against its exact built-in runtime type. This prevents a low-level post-issuance replacement from carrying caller-controlled subtype behavior while preserving identical JSON bytes. Only after that runtime boundary passes does `canonical_json()` snapshot the current canonical bytes, verify that exact snapshot against the external issuance seal, and return the verified snapshot rather than rereading the object. A valid-value rewrite, representation-preserving runtime alias, attempted seal renewal, or missing process-local issuance evidence fails closed. This mechanism is defense-in-depth for in-process misuse only; durable cross-process uniqueness, purpose authorization, and immutable audit/outbox remain responsibilities of the authoritative host or persistence boundary. + +The ordinary representation is fully redacted as `SelectionOutcomeMonitoringPlan()`, so routine logs and assertion failures do not expose tenant, Job, actor, policy, snapshot, or statistical-plan correlations. Canonical JSON remains the explicit evidence serialization boundary. UUID-backed correlations are value-minimized metadata, not anonymous data, and remain subject to purpose-bound authorization, least privilege, retention/export controls, and audit. + +## Governance boundary + +The packet always remains `requires_human_review`, requires explicit human confirmation, and fixes decision authority to `human_review_only`. Its analysis scope is the total selection process for one Job. It does not calculate selection rates, apply the four-fifths rule, estimate statistical significance, infer discrimination, or make an employment-process change. + +Different requester/reviewer references are only an early syntactic guard. Before review, the host must re-resolve **every packet reference** within the exact `tenant_record_id` through its authoritative boundary so a syntactically valid reference from another tenant cannot be mixed into the monitoring envelope. It must specifically re-resolve `actor_reference` and `reviewer_reference` and prove their resolved actor identities are distinct, then verify Job scope, aggregate population completeness, protected-attribute handling, small-sample policy, and statistical-plan provenance before routing the evidence to an authorized analyst and accountable human reviewer for any process change or legal conclusion. UUIDv4 constrains packet-owned trust-reference opacity only; it does not establish tenant ownership, actor identity, or evidence validity. Tenant UUID generation/privacy policy remains owned by the authoritative HRIS boundary. + +## Example + +```python +from datetime import date, datetime, timezone + +from orgmetra_selection_monitoring import build_selection_outcome_monitoring_plan + +plan = build_selection_outcome_monitoring_plan( + tenant_record_id="11111111-1111-4111-8111-111111111111", + monitoring_plan_reference="selection_monitoring_plan:10000000-0000-4000-8000-000000000001", + job_profile_reference="job_profile:10000000-0000-4000-8000-000000000002", + selection_process_reference="selection_process:10000000-0000-4000-8000-000000000003", + population_snapshot_reference="population_snapshot:10000000-0000-4000-8000-000000000004", + population_snapshot_digest="a" * 64, + outcome_snapshot_reference="selection_outcome_snapshot:10000000-0000-4000-8000-000000000005", + outcome_snapshot_digest="b" * 64, + protected_attribute_policy_reference="protected_attribute_policy:10000000-0000-4000-8000-000000000006", + protected_attribute_policy_digest="c" * 64, + small_sample_policy_reference="small_sample_policy:10000000-0000-4000-8000-000000000007", + small_sample_policy_digest="d" * 64, + statistical_plan_reference="statistical_plan:10000000-0000-4000-8000-000000000008", + statistical_plan_digest="e" * 64, + actor_reference="actor:10000000-0000-4000-8000-000000000009", + reviewer_reference="actor:10000000-0000-4000-8000-00000000000a", + monitoring_start=date(2026, 1, 1), + monitoring_end=date(2026, 3, 31), + purpose_code="selection_outcome_monitoring", + reason_code="quarterly_selection_governance", + generated_at=datetime(2026, 4, 2, 8, 30, tzinfo=timezone.utc), + evidence_version=1, +) +``` + +This package writes no database tables and performs no cross-service SQL. A future persistence or analytics implementation must preserve purpose-bound authorization, authoritative tenant-scoped reference and actor resolution, aggregate-only/minimum-necessary access, small-sample controls, immutable audit evidence, and accountable human review independently. diff --git a/packages/selection-monitoring/pyproject.toml b/packages/selection-monitoring/pyproject.toml new file mode 100644 index 000000000..016e22d60 --- /dev/null +++ b/packages/selection-monitoring/pyproject.toml @@ -0,0 +1,24 @@ +[build-system] +requires = ["setuptools>=69"] +build-backend = "setuptools.build_meta" + +[project] +name = "orgmetra-selection-monitoring" +version = "0.1.0" +description = "Governed selection-outcome monitoring plan evidence for Orgmetra." +requires-python = ">=3.12" + +[project.optional-dependencies] +test = ["pytest>=8.3", "pytest-cov>=5.0"] + +[tool.setuptools.packages.find] +where = ["src"] + +[tool.pytest.ini_options] +testpaths = ["tests"] +addopts = [ + "--cov=orgmetra_selection_monitoring", + "--cov-branch", + "--cov-report=term-missing", + "--cov-fail-under=100", +] diff --git a/packages/selection-monitoring/src/orgmetra_selection_monitoring/__init__.py b/packages/selection-monitoring/src/orgmetra_selection_monitoring/__init__.py new file mode 100644 index 000000000..efe1d1382 --- /dev/null +++ b/packages/selection-monitoring/src/orgmetra_selection_monitoring/__init__.py @@ -0,0 +1,8 @@ +"""Public contract for governed selection-outcome monitoring evidence.""" + +from .plan import SelectionOutcomeMonitoringPlan, build_selection_outcome_monitoring_plan + +__all__ = [ + "SelectionOutcomeMonitoringPlan", + "build_selection_outcome_monitoring_plan", +] diff --git a/packages/selection-monitoring/src/orgmetra_selection_monitoring/plan.py b/packages/selection-monitoring/src/orgmetra_selection_monitoring/plan.py new file mode 100644 index 000000000..99bb20b30 --- /dev/null +++ b/packages/selection-monitoring/src/orgmetra_selection_monitoring/plan.py @@ -0,0 +1,404 @@ +"""Governed, aggregate-only selection-outcome monitoring plan evidence. + +The packet binds one Job-scoped total selection process to exact aggregate snapshot, +protected-attribute handling, small-sample interpretation, and statistical-plan evidence. +It carries no candidate identities, protected-attribute values, scores, or decisions and +does not itself compute or assert adverse impact. +""" +from __future__ import annotations + +from dataclasses import dataclass +from datetime import date, datetime, timedelta, timezone +from hashlib import sha256 +import hmac +import json +import re +import secrets +from threading import RLock +from uuid import UUID +from weakref import WeakValueDictionary, finalize + +_CODE_PATTERN = re.compile(r"^[a-z][a-z0-9]*(?:_[a-z0-9]+)+$") +_DIGEST_PATTERN = re.compile(r"^[0-9a-f]{64}$") +_REFERENCE_PATTERN = re.compile( + r"^[a-z][a-z0-9_]{1,31}:[A-Za-z0-9](?:[A-Za-z0-9._-]{0,126}[A-Za-z0-9])?$" +) +_PURPOSE_CODE = "selection_outcome_monitoring" +_ANALYSIS_SCOPE = "total_selection_process_by_job" +_REVIEW_STATE = "requires_human_review" +_DECISION_AUTHORITY = "human_review_only" +_ALLOWED_REASON_CODES = frozenset({"quarterly_selection_governance"}) +_NEXT_ACTION = ( + "Within tenant_record_id, re-resolve every packet reference through its authoritative " + "boundary; specifically re-resolve actor_reference and reviewer_reference and verify " + "their resolved actor identities are distinct; then verify Job scope, aggregate " + "population completeness, protected-attribute handling, small-sample policy, and " + "statistical-plan provenance before submitting the aggregate evidence to an authorized " + "analyst and accountable human reviewer for any employment-process change or legal " + "conclusion." +) +_PROCESS_PLAN_SEAL_KEY = secrets.token_bytes(32) +_PLAN_SEALS: dict[int, str] = {} +_ISSUED_PLAN_IDENTITIES: WeakValueDictionary[int, object] = WeakValueDictionary() +_PLAN_SEALS_LOCK = RLock() +_CANONICAL_TEXT_FIELDS = ( + "tenant_record_id", + "monitoring_plan_reference", + "job_profile_reference", + "selection_process_reference", + "population_snapshot_reference", + "population_snapshot_digest", + "outcome_snapshot_reference", + "outcome_snapshot_digest", + "protected_attribute_policy_reference", + "protected_attribute_policy_digest", + "small_sample_policy_reference", + "small_sample_policy_digest", + "statistical_plan_reference", + "statistical_plan_digest", + "actor_reference", + "reviewer_reference", + "purpose_code", + "reason_code", + "analysis_scope", + "decision_authority", + "review_state", + "next_action", +) +_CANONICAL_EXACT_TYPE_FIELDS = ( + ("monitoring_start", date), + ("monitoring_end", date), + ("generated_at", datetime), + ("evidence_version", int), + ("contains_individual_records", bool), + ("human_confirmation_required", bool), +) + + +def _discard_plan_seal(plan_id: int) -> None: + """Discard process-local seal bytes without resetting live issuance identity.""" + with _PLAN_SEALS_LOCK: + _PLAN_SEALS.pop(plan_id, None) + + +def _register_plan_seal(plan: object, seal: str) -> None: + """Atomically bind one live monitoring-plan identity to one issuance seal.""" + plan_id = id(plan) + with _PLAN_SEALS_LOCK: + if _ISSUED_PLAN_IDENTITIES.get(plan_id) is plan: + raise ValueError("selection monitoring plan issuance evidence already exists") + _PLAN_SEALS[plan_id] = seal + _ISSUED_PLAN_IDENTITIES[plan_id] = plan + finalize(plan, _discard_plan_seal, plan_id) + + +def _authoritative_plan_seal(plan: object) -> str | None: + """Return process-local issuance evidence without trusting plan-owned state.""" + with _PLAN_SEALS_LOCK: + return _PLAN_SEALS.get(id(plan)) + + +def _seal_plan(payload_json: str) -> str: + """Bind one process-local issuance to its exact canonical monitoring-plan bytes.""" + return hmac.new(_PROCESS_PLAN_SEAL_KEY, payload_json.encode("utf-8"), "sha256").hexdigest() + + +def _validate_operational_uuid(value: str, field_name: str) -> None: + """Require canonical non-sentinel UUID text owned by the authoritative HRIS.""" + if type(value) is not str: + raise ValueError(f"{field_name} must be canonical UUID text") + try: + parsed = UUID(value) + except (ValueError, AttributeError, TypeError) as exc: + raise ValueError(f"{field_name} must be canonical UUID text") from exc + if str(parsed) != value or parsed.int in (0, (1 << 128) - 1): + raise ValueError(f"{field_name} must be a canonical operational UUID") + + +def _validate_code(value: str, field_name: str) -> None: + """Require exact bounded descriptive lower snake_case governance text.""" + if type(value) is not str or len(value) > 64 or not _CODE_PATTERN.fullmatch(value): + raise ValueError(f"{field_name} must be bounded two-or-more-word lower snake_case") + + +def _validate_reference(value: str, prefix: str, field_name: str) -> None: + """Require an expected namespace plus a canonical opaque UUIDv4 suffix.""" + error_message = f"{field_name} must be an opaque {prefix}: reference" + if ( + type(value) is not str + or len(value) > 160 + or not _REFERENCE_PATTERN.fullmatch(value) + or not value.startswith(f"{prefix}:") + ): + raise ValueError(error_message) + suffix = value.split(":", 1)[1] + try: + parsed = UUID(suffix) + except (ValueError, AttributeError, TypeError) as exc: + raise ValueError(error_message) from exc + if str(parsed) != suffix or parsed.version != 4 or parsed.int in (0, (1 << 128) - 1): + raise ValueError(error_message) + + +def _validate_digest(value: str, field_name: str) -> None: + """Require exact built-in lowercase SHA-256 hexadecimal evidence.""" + if type(value) is not str or not _DIGEST_PATTERN.fullmatch(value): + raise ValueError(f"{field_name} must be lowercase SHA-256 hex") + + +def _freeze_timestamp(value: datetime) -> datetime: + """Resolve caller timezone behavior once and store one immutable UTC instant.""" + if type(value) is not datetime or value.tzinfo is None: + raise ValueError("generated_at must be an exact timezone-aware datetime") + try: + offset = value.utcoffset() + except Exception as exc: + raise ValueError("generated_at must be an exact timezone-aware datetime") from exc + if type(offset) is not timedelta: + raise ValueError("generated_at must be an exact timezone-aware datetime") + try: + frozen = (value.replace(tzinfo=None) - offset).replace(tzinfo=timezone.utc) + except (OverflowError, ValueError) as exc: + raise ValueError("generated_at must be an exact timezone-aware datetime") from exc + if frozen > datetime.now(timezone.utc): + raise ValueError("generated_at must not be in the future") + return frozen + + +def _canonical_timestamp(value: datetime) -> str: + """Render one already-frozen UTC instant as precision-preserving RFC 3339 text.""" + if type(value) is not datetime or value.tzinfo is not timezone.utc: + raise ValueError("generated_at must be an exact timezone-aware datetime") + return value.isoformat().replace("+00:00", "Z") + + +def _assert_canonical_runtime_evidence(plan: "SelectionOutcomeMonitoringPlan") -> None: + """Reject executable aliases before canonical serialization can touch them.""" + for field_name in _CANONICAL_TEXT_FIELDS: + if type(getattr(plan, field_name)) is not str: + raise ValueError(f"{field_name} runtime evidence must remain exact built-in text") + for field_name, expected_type in _CANONICAL_EXACT_TYPE_FIELDS: + if type(getattr(plan, field_name)) is not expected_type: + raise ValueError( + f"{field_name} runtime evidence must remain exact built-in {expected_type.__name__}" + ) + + +@dataclass(frozen=True, slots=True, repr=False, weakref_slot=True) +class SelectionOutcomeMonitoringPlan: + """Immutable aggregate-monitoring plan awaiting accountable human review.""" + + tenant_record_id: str + monitoring_plan_reference: str + job_profile_reference: str + selection_process_reference: str + population_snapshot_reference: str + population_snapshot_digest: str + outcome_snapshot_reference: str + outcome_snapshot_digest: str + protected_attribute_policy_reference: str + protected_attribute_policy_digest: str + small_sample_policy_reference: str + small_sample_policy_digest: str + statistical_plan_reference: str + statistical_plan_digest: str + actor_reference: str + reviewer_reference: str + monitoring_start: date + monitoring_end: date + purpose_code: str + reason_code: str + generated_at: datetime + evidence_version: int = 1 + analysis_scope: str = _ANALYSIS_SCOPE + contains_individual_records: bool = False + human_confirmation_required: bool = True + decision_authority: str = _DECISION_AUTHORITY + review_state: str = _REVIEW_STATE + next_action: str = _NEXT_ACTION + + def __post_init__(self) -> None: + """Fail closed when direct construction drifts from the governed contract.""" + _validate_operational_uuid(self.tenant_record_id, "tenant_record_id") + _validate_reference( + self.monitoring_plan_reference, + "selection_monitoring_plan", + "monitoring_plan_reference", + ) + _validate_reference(self.job_profile_reference, "job_profile", "job_profile_reference") + _validate_reference( + self.selection_process_reference, + "selection_process", + "selection_process_reference", + ) + _validate_reference( + self.population_snapshot_reference, + "population_snapshot", + "population_snapshot_reference", + ) + _validate_digest(self.population_snapshot_digest, "population_snapshot_digest") + _validate_reference( + self.outcome_snapshot_reference, + "selection_outcome_snapshot", + "outcome_snapshot_reference", + ) + _validate_digest(self.outcome_snapshot_digest, "outcome_snapshot_digest") + _validate_reference( + self.protected_attribute_policy_reference, + "protected_attribute_policy", + "protected_attribute_policy_reference", + ) + _validate_digest( + self.protected_attribute_policy_digest, + "protected_attribute_policy_digest", + ) + _validate_reference( + self.small_sample_policy_reference, + "small_sample_policy", + "small_sample_policy_reference", + ) + _validate_digest(self.small_sample_policy_digest, "small_sample_policy_digest") + _validate_reference( + self.statistical_plan_reference, + "statistical_plan", + "statistical_plan_reference", + ) + _validate_digest(self.statistical_plan_digest, "statistical_plan_digest") + _validate_reference(self.actor_reference, "actor", "actor_reference") + _validate_reference(self.reviewer_reference, "actor", "reviewer_reference") + if self.actor_reference == self.reviewer_reference: + raise ValueError("reviewer_reference must identify a different accountable actor") + if type(self.monitoring_start) is not date: + raise ValueError("monitoring_start must be a calendar date") + if type(self.monitoring_end) is not date: + raise ValueError("monitoring_end must be a calendar date") + if self.monitoring_end < self.monitoring_start: + raise ValueError("monitoring_end must not precede monitoring_start") + _validate_code(self.purpose_code, "purpose_code") + if self.purpose_code != _PURPOSE_CODE: + raise ValueError("purpose_code must remain selection_outcome_monitoring") + _validate_code(self.reason_code, "reason_code") + if self.reason_code not in _ALLOWED_REASON_CODES: + raise ValueError("reason_code must use a reviewed non-sensitive monitoring reason") + object.__setattr__(self, "generated_at", _freeze_timestamp(self.generated_at)) + if type(self.evidence_version) is not int or not 1 <= self.evidence_version <= 2_147_483_647: + raise ValueError("evidence_version must be an integer from 1 through 2147483647") + if type(self.analysis_scope) is not str or self.analysis_scope != _ANALYSIS_SCOPE: + raise ValueError("analysis_scope must remain total_selection_process_by_job") + if self.contains_individual_records is not False: + raise ValueError("monitoring plan must remain aggregate-only") + if self.human_confirmation_required is not True: + raise ValueError("human confirmation is mandatory before monitoring use") + if type(self.decision_authority) is not str or self.decision_authority != _DECISION_AUTHORITY: + raise ValueError("decision_authority must remain human_review_only") + if type(self.review_state) is not str or self.review_state != _REVIEW_STATE: + raise ValueError("review_state must remain requires_human_review") + if type(self.next_action) is not str or self.next_action != _NEXT_ACTION: + raise ValueError("next_action must remain the governed monitoring instruction") + _register_plan_seal(self, _seal_plan(_canonical_plan_json_unchecked(self))) + + def __repr__(self) -> str: + """Return a fully redacted representation safe for routine logs and assertions.""" + return "SelectionOutcomeMonitoringPlan()" + + def canonical_json(self) -> str: + """Return issuance-verified deterministic JSON for immutable audit correlation.""" + payload_json = _canonical_plan_json_unchecked(self) + authoritative_seal = _authoritative_plan_seal(self) + if authoritative_seal is None: + raise ValueError("selection monitoring plan issuance evidence is unavailable") + if not hmac.compare_digest(authoritative_seal, _seal_plan(payload_json)): + raise ValueError("selection monitoring plan evidence changed after issuance") + return payload_json + + def sha256_digest(self) -> str: + """Return SHA-256 over the exact issuance-verified UTF-8 monitoring plan.""" + return sha256(self.canonical_json().encode("utf-8")).hexdigest() + + +def _canonical_plan_json_unchecked(plan: SelectionOutcomeMonitoringPlan) -> str: + """Render canonical bytes without consulting process-local issuance state.""" + _assert_canonical_runtime_evidence(plan) + payload = { + "actor_reference": plan.actor_reference, + "analysis_scope": plan.analysis_scope, + "contains_individual_records": plan.contains_individual_records, + "decision_authority": plan.decision_authority, + "evidence_version": plan.evidence_version, + "generated_at": _canonical_timestamp(plan.generated_at), + "human_confirmation_required": plan.human_confirmation_required, + "job_profile_reference": plan.job_profile_reference, + "monitoring_end": plan.monitoring_end.isoformat(), + "monitoring_plan_reference": plan.monitoring_plan_reference, + "monitoring_start": plan.monitoring_start.isoformat(), + "next_action": plan.next_action, + "outcome_snapshot_digest": plan.outcome_snapshot_digest, + "outcome_snapshot_reference": plan.outcome_snapshot_reference, + "population_snapshot_digest": plan.population_snapshot_digest, + "population_snapshot_reference": plan.population_snapshot_reference, + "protected_attribute_policy_digest": plan.protected_attribute_policy_digest, + "protected_attribute_policy_reference": plan.protected_attribute_policy_reference, + "purpose_code": plan.purpose_code, + "reason_code": plan.reason_code, + "review_state": plan.review_state, + "reviewer_reference": plan.reviewer_reference, + "selection_process_reference": plan.selection_process_reference, + "small_sample_policy_digest": plan.small_sample_policy_digest, + "small_sample_policy_reference": plan.small_sample_policy_reference, + "statistical_plan_digest": plan.statistical_plan_digest, + "statistical_plan_reference": plan.statistical_plan_reference, + "tenant_record_id": plan.tenant_record_id, + } + return json.dumps(payload, sort_keys=True, separators=(",", ":"), ensure_ascii=True) + + +def build_selection_outcome_monitoring_plan( + *, + tenant_record_id: str, + monitoring_plan_reference: str, + job_profile_reference: str, + selection_process_reference: str, + population_snapshot_reference: str, + population_snapshot_digest: str, + outcome_snapshot_reference: str, + outcome_snapshot_digest: str, + protected_attribute_policy_reference: str, + protected_attribute_policy_digest: str, + small_sample_policy_reference: str, + small_sample_policy_digest: str, + statistical_plan_reference: str, + statistical_plan_digest: str, + actor_reference: str, + reviewer_reference: str, + monitoring_start: date, + monitoring_end: date, + purpose_code: str, + reason_code: str, + generated_at: datetime, + evidence_version: int = 1, +) -> SelectionOutcomeMonitoringPlan: + """Build an aggregate-only monitoring plan pending accountable human review.""" + return SelectionOutcomeMonitoringPlan( + tenant_record_id=tenant_record_id, + monitoring_plan_reference=monitoring_plan_reference, + job_profile_reference=job_profile_reference, + selection_process_reference=selection_process_reference, + population_snapshot_reference=population_snapshot_reference, + population_snapshot_digest=population_snapshot_digest, + outcome_snapshot_reference=outcome_snapshot_reference, + outcome_snapshot_digest=outcome_snapshot_digest, + protected_attribute_policy_reference=protected_attribute_policy_reference, + protected_attribute_policy_digest=protected_attribute_policy_digest, + small_sample_policy_reference=small_sample_policy_reference, + small_sample_policy_digest=small_sample_policy_digest, + statistical_plan_reference=statistical_plan_reference, + statistical_plan_digest=statistical_plan_digest, + actor_reference=actor_reference, + reviewer_reference=reviewer_reference, + monitoring_start=monitoring_start, + monitoring_end=monitoring_end, + purpose_code=purpose_code, + reason_code=reason_code, + generated_at=generated_at, + evidence_version=evidence_version, + ) diff --git a/packages/selection-monitoring/tests/test_actor_separation.py b/packages/selection-monitoring/tests/test_actor_separation.py new file mode 100644 index 000000000..a1f5a4534 --- /dev/null +++ b/packages/selection-monitoring/tests/test_actor_separation.py @@ -0,0 +1,62 @@ +"""Actor-separation and tenant-scope regressions for selection-monitoring evidence.""" + +from __future__ import annotations + +from datetime import date, datetime, timezone + +import pytest + +from orgmetra_selection_monitoring import build_selection_outcome_monitoring_plan + + +def _build(**overrides): + """Build a valid monitoring plan with canonical opaque references.""" + values = { + "tenant_record_id": "11111111-1111-4111-8111-111111111111", + "monitoring_plan_reference": "selection_monitoring_plan:10000000-0000-4000-8000-000000000001", + "job_profile_reference": "job_profile:10000000-0000-4000-8000-000000000002", + "selection_process_reference": "selection_process:10000000-0000-4000-8000-000000000003", + "population_snapshot_reference": "population_snapshot:10000000-0000-4000-8000-000000000004", + "population_snapshot_digest": "a" * 64, + "outcome_snapshot_reference": "selection_outcome_snapshot:10000000-0000-4000-8000-000000000005", + "outcome_snapshot_digest": "b" * 64, + "protected_attribute_policy_reference": "protected_attribute_policy:10000000-0000-4000-8000-000000000006", + "protected_attribute_policy_digest": "c" * 64, + "small_sample_policy_reference": "small_sample_policy:10000000-0000-4000-8000-000000000007", + "small_sample_policy_digest": "d" * 64, + "statistical_plan_reference": "statistical_plan:10000000-0000-4000-8000-000000000008", + "statistical_plan_digest": "e" * 64, + "actor_reference": "actor:10000000-0000-4000-8000-000000000009", + "reviewer_reference": "actor:10000000-0000-4000-8000-00000000000a", + "monitoring_start": date(2026, 1, 1), + "monitoring_end": date(2026, 3, 31), + "purpose_code": "selection_outcome_monitoring", + "reason_code": "quarterly_selection_governance", + "generated_at": datetime(2026, 4, 2, 8, 30, tzinfo=timezone.utc), + } + values.update(overrides) + return build_selection_outcome_monitoring_plan(**values) + + +def test_requester_and_reviewer_require_authoritative_actor_separation() -> None: + """Require both syntactic and authoritative actor separation before review.""" + with pytest.raises(ValueError, match="different accountable actor"): + _build(reviewer_reference="actor:10000000-0000-4000-8000-000000000009") + + normalized_next_action = _build().next_action.lower() + assert "actor_reference and reviewer_reference" in normalized_next_action + assert "resolved actor identities are distinct" in normalized_next_action + + +def test_review_requires_every_reference_to_resolve_in_the_exact_tenant() -> None: + """Prevent cross-tenant evidence mixing behind otherwise valid opaque references.""" + action = _build().next_action + tenant_clause = "Within tenant_record_id, re-resolve every packet reference" + actor_clause = "verify their resolved actor identities are distinct" + job_clause = "verify Job scope" + review_clause = "accountable human reviewer" + + assert tenant_clause in action + assert action.index(tenant_clause) < action.index(actor_clause) + assert action.index(actor_clause) < action.index(job_clause) + assert action.index(job_clause) < action.index(review_clause) diff --git a/packages/selection-monitoring/tests/test_evidence_version.py b/packages/selection-monitoring/tests/test_evidence_version.py new file mode 100644 index 000000000..30c0ca596 --- /dev/null +++ b/packages/selection-monitoring/tests/test_evidence_version.py @@ -0,0 +1,39 @@ +"""Regression coverage for explicit selection-monitoring evidence versioning.""" + +from dataclasses import replace +import json + +import pytest + +from orgmetra_selection_monitoring import SelectionOutcomeMonitoringPlan + +from test_plan import valid_kwargs + + +def _plan(evidence_version: int = 1) -> SelectionOutcomeMonitoringPlan: + """Build one valid monitoring plan while varying only its evidence version.""" + kwargs = valid_kwargs() + kwargs["evidence_version"] = evidence_version + return SelectionOutcomeMonitoringPlan(**kwargs) + + +def test_evidence_version_is_part_of_immutable_monitoring_evidence() -> None: + """Bind evidence revision identity into canonical JSON and the packet digest.""" + first = _plan(1) + second = _plan(2) + assert json.loads(first.canonical_json())["evidence_version"] == 1 + assert first.canonical_json() != second.canonical_json() + assert first.sha256_digest() != second.sha256_digest() + + +@pytest.mark.parametrize("evidence_version", [True, False, 0, -1, 2_147_483_648, "1", 1.0]) +def test_rejects_noncanonical_evidence_versions(evidence_version: object) -> None: + """Reject booleans, non-integers, non-positive values, and signed-int32 overflow.""" + with pytest.raises(ValueError, match="evidence_version"): + _plan(evidence_version) # type: ignore[arg-type] + + +def test_replace_cannot_bypass_monitoring_evidence_version_validation() -> None: + """Revalidate explicit evidence-version bounds when immutable plans are copied.""" + with pytest.raises(ValueError, match="evidence_version"): + replace(_plan(), evidence_version=0) diff --git a/packages/selection-monitoring/tests/test_fixed_governance_runtime_integrity.py b/packages/selection-monitoring/tests/test_fixed_governance_runtime_integrity.py new file mode 100644 index 000000000..13732fdeb --- /dev/null +++ b/packages/selection-monitoring/tests/test_fixed_governance_runtime_integrity.py @@ -0,0 +1,32 @@ +"""Regression tests for fixed selection-monitoring governance text integrity.""" + +from dataclasses import replace + +import pytest + +from test_plan import build_valid + + +class ForgedFixedGovernanceText(str): + """String subclass that lies during equality checks but keeps forged JSON text.""" + + def __eq__(self, other: object) -> bool: + """Pretend the forged value equals every governed constant.""" + return True + + def __ne__(self, other: object) -> bool: + """Pretend the forged value never differs from a governed constant.""" + return False + + +@pytest.mark.parametrize( + "field_name", + ["analysis_scope", "decision_authority", "review_state", "next_action"], +) +def test_rejects_string_subclasses_for_fixed_governance_fields(field_name: str) -> None: + """Reject canonical evidence whose fixed governance text can bypass comparison.""" + with pytest.raises(ValueError, match=field_name): + replace( + build_valid(), + **{field_name: ForgedFixedGovernanceText("forged_governance_value")}, + ) diff --git a/packages/selection-monitoring/tests/test_issuance_integrity.py b/packages/selection-monitoring/tests/test_issuance_integrity.py new file mode 100644 index 000000000..23a767a64 --- /dev/null +++ b/packages/selection-monitoring/tests/test_issuance_integrity.py @@ -0,0 +1,108 @@ +"""Regression tests for immutable selection-monitoring issuance evidence.""" + +from __future__ import annotations + +import copy +from datetime import date, datetime, timezone +import pickle + +import pytest + +from orgmetra_selection_monitoring import build_selection_outcome_monitoring_plan +from orgmetra_selection_monitoring import plan as plan_module + + +_BASE_KWARGS: dict[str, object] = { + "tenant_record_id": "11111111-1111-4111-8111-111111111111", + "monitoring_plan_reference": "selection_monitoring_plan:10000000-0000-4000-8000-000000000001", + "job_profile_reference": "job_profile:10000000-0000-4000-8000-000000000002", + "selection_process_reference": "selection_process:10000000-0000-4000-8000-000000000003", + "population_snapshot_reference": "population_snapshot:10000000-0000-4000-8000-000000000004", + "population_snapshot_digest": "a" * 64, + "outcome_snapshot_reference": "selection_outcome_snapshot:10000000-0000-4000-8000-000000000005", + "outcome_snapshot_digest": "b" * 64, + "protected_attribute_policy_reference": "protected_attribute_policy:10000000-0000-4000-8000-000000000006", + "protected_attribute_policy_digest": "c" * 64, + "small_sample_policy_reference": "small_sample_policy:10000000-0000-4000-8000-000000000007", + "small_sample_policy_digest": "d" * 64, + "statistical_plan_reference": "statistical_plan:10000000-0000-4000-8000-000000000008", + "statistical_plan_digest": "e" * 64, + "actor_reference": "actor:10000000-0000-4000-8000-000000000009", + "reviewer_reference": "actor:10000000-0000-4000-8000-00000000000a", + "monitoring_start": date(2026, 1, 1), + "monitoring_end": date(2026, 3, 31), + "purpose_code": "selection_outcome_monitoring", + "reason_code": "quarterly_selection_governance", + "generated_at": datetime(2026, 4, 2, 8, 30, 0, 123456, tzinfo=timezone.utc), +} + + +def _build_plan(): + """Build one valid issued monitoring plan for tamper-evidence regressions.""" + return build_selection_outcome_monitoring_plan(**_BASE_KWARGS) + + +def test_post_issuance_rewrite_cannot_change_canonical_evidence() -> None: + """Reject a valid-value rewrite after the governed plan has been issued.""" + plan = _build_plan() + original = plan.canonical_json() + + object.__setattr__(plan, "population_snapshot_digest", "f" * 64) + + with pytest.raises(ValueError, match="changed after issuance"): + plan.canonical_json() + with pytest.raises(ValueError, match="changed after issuance"): + plan.sha256_digest() + assert original != plan_module._canonical_plan_json_unchecked(plan) + + +def test_missing_process_local_issuance_evidence_fails_closed() -> None: + """Reject canonical export when process-local issuance evidence is unavailable.""" + plan = _build_plan() + plan_module._discard_plan_seal(id(plan)) + + with pytest.raises(ValueError, match="issuance evidence is unavailable"): + plan.canonical_json() + + +def test_copied_or_serialized_plan_fails_closed_without_issuance_evidence() -> None: + """Reject copies and serialized plans that bypass the original issuance seal.""" + plan = _build_plan() + clones = ( + copy.copy(plan), + copy.deepcopy(plan), + pickle.loads(pickle.dumps(plan)), + ) + + for clone in clones: + with pytest.raises(ValueError, match="issuance evidence is unavailable"): + clone.canonical_json() + + +def test_reinitialization_cannot_renew_issuance_evidence_after_valid_value_rewrite() -> None: + """Keep one live plan identity bound to its original construction evidence.""" + plan = _build_plan() + original = plan.canonical_json() + + object.__setattr__(plan, "population_snapshot_digest", "f" * 64) + + with pytest.raises(ValueError, match="issuance evidence already exists"): + plan.__post_init__() + with pytest.raises(ValueError, match="changed after issuance"): + plan.canonical_json() + assert original != plan_module._canonical_plan_json_unchecked(plan) + + +def test_discarded_seal_cannot_be_renewed_after_valid_value_rewrite() -> None: + """Do not let seal loss reset the live plan's single-use issuance lifecycle.""" + plan = _build_plan() + original = plan.canonical_json() + + plan_module._discard_plan_seal(id(plan)) + object.__setattr__(plan, "population_snapshot_digest", "f" * 64) + + with pytest.raises(ValueError, match="issuance evidence already exists"): + plan.__post_init__() + with pytest.raises(ValueError, match="issuance evidence is unavailable"): + plan.canonical_json() + assert original != plan_module._canonical_plan_json_unchecked(plan) diff --git a/packages/selection-monitoring/tests/test_plan.py b/packages/selection-monitoring/tests/test_plan.py new file mode 100644 index 000000000..f9a14e101 --- /dev/null +++ b/packages/selection-monitoring/tests/test_plan.py @@ -0,0 +1,306 @@ +"""Executable contract tests for governed selection-outcome monitoring.""" + +from __future__ import annotations + +from dataclasses import FrozenInstanceError, replace +from datetime import date, datetime, timedelta, timezone, tzinfo +from hashlib import sha256 +import json + +import pytest + +from orgmetra_selection_monitoring import ( + SelectionOutcomeMonitoringPlan, + build_selection_outcome_monitoring_plan, +) + + +DIGEST_A = "a" * 64 +DIGEST_B = "b" * 64 +DIGEST_C = "c" * 64 +DIGEST_D = "d" * 64 +DIGEST_E = "e" * 64 + + +def valid_kwargs() -> dict[str, object]: + """Return one valid monitoring-plan input using opaque UUID references.""" + return { + "tenant_record_id": "11111111-1111-4111-8111-111111111111", + "monitoring_plan_reference": "selection_monitoring_plan:10000000-0000-4000-8000-000000000001", + "job_profile_reference": "job_profile:10000000-0000-4000-8000-000000000002", + "selection_process_reference": "selection_process:10000000-0000-4000-8000-000000000003", + "population_snapshot_reference": "population_snapshot:10000000-0000-4000-8000-000000000004", + "population_snapshot_digest": DIGEST_A, + "outcome_snapshot_reference": "selection_outcome_snapshot:10000000-0000-4000-8000-000000000005", + "outcome_snapshot_digest": DIGEST_B, + "protected_attribute_policy_reference": "protected_attribute_policy:10000000-0000-4000-8000-000000000006", + "protected_attribute_policy_digest": DIGEST_C, + "small_sample_policy_reference": "small_sample_policy:10000000-0000-4000-8000-000000000007", + "small_sample_policy_digest": DIGEST_D, + "statistical_plan_reference": "statistical_plan:10000000-0000-4000-8000-000000000008", + "statistical_plan_digest": DIGEST_E, + "actor_reference": "actor:10000000-0000-4000-8000-000000000009", + "reviewer_reference": "actor:10000000-0000-4000-8000-00000000000a", + "monitoring_start": date(2026, 1, 1), + "monitoring_end": date(2026, 3, 31), + "purpose_code": "selection_outcome_monitoring", + "reason_code": "quarterly_selection_governance", + "generated_at": datetime(2026, 4, 2, 8, 30, 0, 123456, tzinfo=timezone.utc), + } + + +def build_valid() -> SelectionOutcomeMonitoringPlan: + """Build one valid governed monitoring plan.""" + return build_selection_outcome_monitoring_plan(**valid_kwargs()) + + +def test_builds_aggregate_only_human_review_plan() -> None: + """Keep the packet aggregate-only and human-review-only.""" + plan = build_valid() + + assert plan.analysis_scope == "total_selection_process_by_job" + assert plan.contains_individual_records is False + assert plan.human_confirmation_required is True + assert plan.decision_authority == "human_review_only" + assert plan.review_state == "requires_human_review" + assert "authorized analyst" in plan.next_action + assert "legal conclusion" in plan.next_action + + +def test_canonical_json_and_digest_are_deterministic_and_value_free() -> None: + """Preserve deterministic canonical evidence without individual values.""" + plan = build_valid() + payload = json.loads(plan.canonical_json()) + + assert payload["generated_at"] == "2026-04-02T08:30:00.123456Z" + assert payload["monitoring_start"] == "2026-01-01" + assert payload["monitoring_end"] == "2026-03-31" + assert "candidate" not in payload + assert "protected_attribute_value" not in payload + assert plan.sha256_digest() == sha256(plan.canonical_json().encode("utf-8")).hexdigest() + + +def test_fractional_seconds_remain_distinct_evidence() -> None: + """Keep sub-second evidence instants distinct in canonical evidence.""" + first = build_valid() + second = replace( + first, + generated_at=first.generated_at + timedelta(microseconds=1), + ) + + assert first.canonical_json() != second.canonical_json() + assert first.sha256_digest() != second.sha256_digest() + + +@pytest.mark.parametrize( + ("field_name", "value"), + [ + ("tenant_record_id", "not-a-uuid"), + ("tenant_record_id", "00000000-0000-0000-0000-000000000000"), + ("tenant_record_id", "FFFFFFFF-FFFF-FFFF-FFFF-FFFFFFFFFFFF"), + ("tenant_record_id", None), + ], +) +def test_rejects_nonoperational_tenant_identity(field_name: str, value: object) -> None: + """Reject malformed, sentinel, and noncanonical tenant UUIDs.""" + kwargs = valid_kwargs() + kwargs[field_name] = value + with pytest.raises(ValueError, match="tenant_record_id"): + build_selection_outcome_monitoring_plan(**kwargs) + + +@pytest.mark.parametrize( + ("field_name", "value", "message"), + [ + ("monitoring_plan_reference", "wrong:plan-001", "selection_monitoring_plan"), + ("job_profile_reference", "job:job-001", "job_profile"), + ("selection_process_reference", "selection:process-001", "selection_process"), + ("population_snapshot_reference", "population:population-001", "population_snapshot"), + ("outcome_snapshot_reference", "outcome:outcomes-001", "selection_outcome_snapshot"), + ( + "protected_attribute_policy_reference", + "policy:protected-001", + "protected_attribute_policy", + ), + ("small_sample_policy_reference", "policy:small-001", "small_sample_policy"), + ("statistical_plan_reference", "statistics:plan-001", "statistical_plan"), + ("actor_reference", "person:requester-001", "actor"), + ("reviewer_reference", "reviewer:reviewer-001", "actor"), + ("actor_reference", "actor:", "actor"), + ("actor_reference", 1, "actor"), + ("actor_reference", "actor:" + "a" * 155, "actor"), + ], +) +def test_rejects_bad_opaque_references( + field_name: str, + value: object, + message: str, +) -> None: + """Reject malformed or wrong-namespace opaque references.""" + kwargs = valid_kwargs() + kwargs[field_name] = value + with pytest.raises(ValueError, match=message): + build_selection_outcome_monitoring_plan(**kwargs) + + +@pytest.mark.parametrize( + "field_name", + [ + "population_snapshot_digest", + "outcome_snapshot_digest", + "protected_attribute_policy_digest", + "small_sample_policy_digest", + "statistical_plan_digest", + ], +) +@pytest.mark.parametrize("value", ["A" * 64, "a" * 63, 1]) +def test_rejects_malformed_digests(field_name: str, value: object) -> None: + """Require exact lowercase SHA-256 evidence digests.""" + kwargs = valid_kwargs() + kwargs[field_name] = value + with pytest.raises(ValueError, match="lowercase SHA-256"): + build_selection_outcome_monitoring_plan(**kwargs) + + +def test_reviewer_must_be_distinct_from_requester() -> None: + """Reject identical requester and reviewer references.""" + kwargs = valid_kwargs() + kwargs["reviewer_reference"] = kwargs["actor_reference"] + with pytest.raises(ValueError, match="different accountable actor"): + build_selection_outcome_monitoring_plan(**kwargs) + + +@pytest.mark.parametrize( + ("field_name", "value", "message"), + [ + ("monitoring_start", datetime(2026, 1, 1, tzinfo=timezone.utc), "calendar date"), + ("monitoring_end", datetime(2026, 3, 31, tzinfo=timezone.utc), "calendar date"), + ("monitoring_start", "2026-01-01", "calendar date"), + ("monitoring_end", "2026-03-31", "calendar date"), + ], +) +def test_rejects_non_date_monitoring_bounds( + field_name: str, + value: object, + message: str, +) -> None: + """Require business dates rather than datetimes or text.""" + kwargs = valid_kwargs() + kwargs[field_name] = value + with pytest.raises(ValueError, match=message): + build_selection_outcome_monitoring_plan(**kwargs) + + +def test_rejects_reverse_monitoring_window() -> None: + """Reject a monitoring interval whose end precedes its start.""" + kwargs = valid_kwargs() + kwargs["monitoring_start"] = date(2026, 4, 1) + with pytest.raises(ValueError, match="must not precede"): + build_selection_outcome_monitoring_plan(**kwargs) + + +@pytest.mark.parametrize( + ("field_name", "value", "message"), + [ + ("purpose_code", "selection_review", "selection_outcome_monitoring"), + ("purpose_code", "SelectionOutcomeMonitoring", "lower snake_case"), + ("purpose_code", "a_" + "b" * 64, "lower snake_case"), + ("purpose_code", 1, "lower snake_case"), + ("reason_code", "quarterly", "lower snake_case"), + ("reason_code", "Quarterly_Review", "lower snake_case"), + ("reason_code", 1, "lower snake_case"), + ], +) +def test_rejects_bad_governance_codes( + field_name: str, + value: object, + message: str, +) -> None: + """Require fixed purpose plus bounded descriptive governance codes.""" + kwargs = valid_kwargs() + kwargs[field_name] = value + with pytest.raises(ValueError, match=message): + build_selection_outcome_monitoring_plan(**kwargs) + + +class NullOffsetTz(tzinfo): + """Timezone fixture whose UTC offset is intentionally unknown.""" + + def utcoffset(self, dt: datetime | None) -> None: + """Return no UTC offset.""" + return None + + def dst(self, dt: datetime | None) -> None: + """Return no daylight-saving offset.""" + return None + + def tzname(self, dt: datetime | None) -> str: + """Return a stable fixture timezone label.""" + return "NULL" + + +@pytest.mark.parametrize( + "value", + [ + datetime(2026, 4, 2, 8, 30), + "2026-04-02T08:30:00Z", + 1, + datetime(2026, 4, 2, 8, 30).replace(tzinfo=NullOffsetTz()), + ], +) +def test_rejects_nonaware_generation_time(value: object) -> None: + """Require a timezone-aware evidence-generation instant.""" + kwargs = valid_kwargs() + kwargs["generated_at"] = value + with pytest.raises(ValueError, match="timezone-aware"): + build_selection_outcome_monitoring_plan(**kwargs) + + +@pytest.mark.parametrize( + ("field_name", "value", "message"), + [ + ("analysis_scope", "component_only", "total_selection_process_by_job"), + ("contains_individual_records", True, "aggregate-only"), + ("contains_individual_records", 0, "aggregate-only"), + ("human_confirmation_required", False, "human confirmation"), + ("human_confirmation_required", 1, "human confirmation"), + ("decision_authority", "automated", "human_review_only"), + ("review_state", "approved", "requires_human_review"), + ("next_action", "Compute adverse impact.", "governed monitoring instruction"), + ], +) +def test_direct_constructor_and_replace_fail_closed( + field_name: str, + value: object, + message: str, +) -> None: + """Revalidate immutable governance fields under dataclass replacement.""" + plan = build_valid() + with pytest.raises(ValueError, match=message): + replace(plan, **{field_name: value}) + + +def test_frozen_plan_rejects_mutation() -> None: + """Prevent in-place mutation of a governed monitoring plan.""" + plan = build_valid() + with pytest.raises(FrozenInstanceError): + plan.review_state = "approved" + + +def test_timezone_is_normalized_without_losing_precision() -> None: + """Normalize offsets to UTC while preserving microsecond identity.""" + kwargs = valid_kwargs() + kwargs["generated_at"] = datetime( + 2026, + 4, + 2, + 17, + 30, + 0, + 654321, + tzinfo=timezone(timedelta(hours=9)), + ) + plan = build_selection_outcome_monitoring_plan(**kwargs) + + payload = json.loads(plan.canonical_json()) + assert payload["generated_at"] == "2026-04-02T08:30:00.654321Z" diff --git a/packages/selection-monitoring/tests/test_post_issuance_runtime_type_integrity.py b/packages/selection-monitoring/tests/test_post_issuance_runtime_type_integrity.py new file mode 100644 index 000000000..5340b9fe7 --- /dev/null +++ b/packages/selection-monitoring/tests/test_post_issuance_runtime_type_integrity.py @@ -0,0 +1,51 @@ +"""Regression tests for post-issuance runtime-type integrity.""" + +from __future__ import annotations + +from datetime import date + +import pytest + +from test_plan import build_valid + + +class SameValueText(str): + """Behavior-bearing text that preserves the issued canonical string bytes.""" + + +class ExecutableDate(date): + """Date subtype whose method must not execute during canonical export.""" + + def isoformat(self) -> str: + """Trip if canonical export reaches caller-controlled date behavior.""" + raise AssertionError("caller-controlled date method executed") + + +def test_same_value_text_subclass_cannot_preserve_issuance_authority() -> None: + """Reject a same-byte text alias rather than trusting only the HMAC seal.""" + plan = build_valid() + object.__setattr__( + plan, + "population_snapshot_digest", + SameValueText(plan.population_snapshot_digest), + ) + + with pytest.raises(ValueError, match="runtime evidence"): + plan.canonical_json() + + +def test_date_subclass_is_rejected_before_isoformat_executes() -> None: + """Reject executable date evidence before canonical rendering calls it.""" + plan = build_valid() + object.__setattr__( + plan, + "monitoring_start", + ExecutableDate( + plan.monitoring_start.year, + plan.monitoring_start.month, + plan.monitoring_start.day, + ), + ) + + with pytest.raises(ValueError, match="runtime evidence"): + plan.canonical_json() diff --git a/packages/selection-monitoring/tests/test_privacy.py b/packages/selection-monitoring/tests/test_privacy.py new file mode 100644 index 000000000..85b02f9af --- /dev/null +++ b/packages/selection-monitoring/tests/test_privacy.py @@ -0,0 +1,60 @@ +"""Privacy regressions for aggregate selection-outcome monitoring evidence.""" + +from dataclasses import replace +from datetime import date, datetime, timezone + +import pytest + +from orgmetra_selection_monitoring import build_selection_outcome_monitoring_plan + + +def _build(): + """Build one valid aggregate monitoring plan for privacy-focused assertions.""" + return build_selection_outcome_monitoring_plan( + tenant_record_id="11111111-1111-4111-8111-111111111111", + monitoring_plan_reference="selection_monitoring_plan:10000000-0000-4000-8000-000000000001", + job_profile_reference="job_profile:10000000-0000-4000-8000-000000000002", + selection_process_reference="selection_process:10000000-0000-4000-8000-000000000003", + population_snapshot_reference="population_snapshot:10000000-0000-4000-8000-000000000004", + population_snapshot_digest="a" * 64, + outcome_snapshot_reference="selection_outcome_snapshot:10000000-0000-4000-8000-000000000005", + outcome_snapshot_digest="b" * 64, + protected_attribute_policy_reference="protected_attribute_policy:10000000-0000-4000-8000-000000000006", + protected_attribute_policy_digest="c" * 64, + small_sample_policy_reference="small_sample_policy:10000000-0000-4000-8000-000000000007", + small_sample_policy_digest="d" * 64, + statistical_plan_reference="statistical_plan:10000000-0000-4000-8000-000000000008", + statistical_plan_digest="e" * 64, + actor_reference="actor:10000000-0000-4000-8000-000000000009", + reviewer_reference="actor:10000000-0000-4000-8000-00000000000a", + monitoring_start=date(2026, 1, 1), + monitoring_end=date(2026, 3, 31), + purpose_code="selection_outcome_monitoring", + reason_code="quarterly_selection_governance", + generated_at=datetime(2026, 4, 2, 8, 30, 0, 123456, tzinfo=timezone.utc), + ) + + +@pytest.mark.parametrize( + "reason_code", + ["jane_doe", "salary_120000", "race_gender_review", "candidate_alice_smith"], +) +def test_reason_code_rejects_personal_or_value_bearing_free_form_codes(reason_code: str) -> None: + """Prevent governance reason metadata from becoming an individual-data channel.""" + plan = _build() + with pytest.raises(ValueError): + replace(plan, reason_code=reason_code) + + +def test_repr_redacts_selection_monitoring_correlations() -> None: + """Keep Job, actor, policy, and statistical correlations out of routine repr output.""" + plan = _build() + rendered = repr(plan) + assert rendered == "SelectionOutcomeMonitoringPlan()" + for sensitive in ( + plan.job_profile_reference, + plan.actor_reference, + plan.protected_attribute_policy_reference, + plan.statistical_plan_digest, + ): + assert sensitive not in rendered diff --git a/packages/selection-monitoring/tests/test_quality_workflow_trigger.py b/packages/selection-monitoring/tests/test_quality_workflow_trigger.py new file mode 100644 index 000000000..1b38326ed --- /dev/null +++ b/packages/selection-monitoring/tests/test_quality_workflow_trigger.py @@ -0,0 +1,38 @@ +"""Regression tests for canonical selection-monitoring Foundation ownership.""" + +from pathlib import Path + + +_REPOSITORY_ROOT = Path(__file__).resolve().parents[3] +_FOUNDATION_WORKFLOW = _REPOSITORY_ROOT / ".github/workflows/foundation-ci.yml" +_DEPENDENCY_HYGIENE = _REPOSITORY_ROOT / "tests/test_foundation_ci_dependency_hygiene.sh" +_ARTIFACT_CONTRACT = _REPOSITORY_ROOT / "tests/test_foundation_ci_selection_monitoring_artifact.sh" +_RETIRED_WORKFLOW = _REPOSITORY_ROOT / ".github/workflows/selection-monitoring-quality.yml" +_PYPROJECT = _REPOSITORY_ROOT / "packages/selection-monitoring/pyproject.toml" + + +def _read(path: Path) -> str: + return path.read_text(encoding="utf-8") + + +def test_retired_selection_monitoring_workflow_stays_deleted() -> None: + """Prevent protected workflow consolidation from being reversed by old ancestry.""" + assert not _RETIRED_WORKFLOW.exists() + + +def test_foundation_owns_selection_monitoring_quality_contract() -> None: + """Require canonical Foundation CI to discover and execute the package contract.""" + foundation = _read(_FOUNDATION_WORKFLOW) + hygiene = _read(_DEPENDENCY_HYGIENE) + artifact = _read(_ARTIFACT_CONTRACT) + pyproject = _read(_PYPROJECT) + + assert "bash tests/test_foundation_ci_dependency_hygiene.sh" in foundation + assert '"${repository_root}"/tests/test_foundation_ci_*_artifact.sh' in hygiene + assert 'bash "${artifact_contract}"' in hygiene + assert 'retired_workflow="${repository_root}/.github/workflows/selection-monitoring-quality.yml"' in artifact + assert 'PYTHONPATH="${package_root}/src"' in artifact + assert '-c "${package_root}/pyproject.toml"' in artifact + assert '"${package_root}/tests"' in artifact + assert '"--cov-branch"' in pyproject + assert '"--cov-fail-under=100"' in pyproject diff --git a/packages/selection-monitoring/tests/test_reference_privacy.py b/packages/selection-monitoring/tests/test_reference_privacy.py new file mode 100644 index 000000000..bb9bdf821 --- /dev/null +++ b/packages/selection-monitoring/tests/test_reference_privacy.py @@ -0,0 +1,146 @@ +"""Privacy regressions for selection-monitoring opaque references.""" + +from __future__ import annotations + +from dataclasses import replace +from datetime import date, datetime, timezone + +import pytest + +from orgmetra_selection_monitoring import ( + SelectionOutcomeMonitoringPlan, + build_selection_outcome_monitoring_plan, +) + +UUID1_ID = "6ba7b810-9dad-11d1-80b4-00c04fd430c8" +UUID7_TENANT = "10000000-0000-7000-8000-000000000001" + + +def _valid_values(**overrides) -> dict[str, object]: + """Return valid constructor values for every governed monitoring field.""" + values: dict[str, object] = { + "tenant_record_id": "11111111-1111-4111-8111-111111111111", + "monitoring_plan_reference": "selection_monitoring_plan:10000000-0000-4000-8000-000000000001", + "job_profile_reference": "job_profile:10000000-0000-4000-8000-000000000002", + "selection_process_reference": "selection_process:10000000-0000-4000-8000-000000000003", + "population_snapshot_reference": "population_snapshot:10000000-0000-4000-8000-000000000004", + "population_snapshot_digest": "a" * 64, + "outcome_snapshot_reference": "selection_outcome_snapshot:10000000-0000-4000-8000-000000000005", + "outcome_snapshot_digest": "b" * 64, + "protected_attribute_policy_reference": "protected_attribute_policy:10000000-0000-4000-8000-000000000006", + "protected_attribute_policy_digest": "c" * 64, + "small_sample_policy_reference": "small_sample_policy:10000000-0000-4000-8000-000000000007", + "small_sample_policy_digest": "d" * 64, + "statistical_plan_reference": "statistical_plan:10000000-0000-4000-8000-000000000008", + "statistical_plan_digest": "e" * 64, + "actor_reference": "actor:10000000-0000-4000-8000-000000000009", + "reviewer_reference": "actor:10000000-0000-4000-8000-00000000000a", + "monitoring_start": date(2026, 1, 1), + "monitoring_end": date(2026, 3, 31), + "purpose_code": "selection_outcome_monitoring", + "reason_code": "quarterly_selection_governance", + "generated_at": datetime(2026, 4, 2, 8, 30, tzinfo=timezone.utc), + } + values.update(overrides) + return values + + +def _build(**overrides) -> SelectionOutcomeMonitoringPlan: + """Build a valid packet through the public builder.""" + return build_selection_outcome_monitoring_plan(**_valid_values(**overrides)) # type: ignore[arg-type] + + +def _direct(**overrides) -> SelectionOutcomeMonitoringPlan: + """Construct a packet directly to prove dataclass invariants cannot be bypassed.""" + return SelectionOutcomeMonitoringPlan(**_valid_values(**overrides)) # type: ignore[arg-type] + + +def test_authoritative_uuid7_tenant_identity_is_accepted_by_all_construction_paths() -> None: + """The monitoring leaf must accept tenant UUIDs already valid in authoritative core.""" + packet = _build(tenant_record_id=UUID7_TENANT) + replaced = replace(_build(), tenant_record_id=UUID7_TENANT) + direct = _direct(tenant_record_id=UUID7_TENANT) + + assert packet.tenant_record_id == UUID7_TENANT + assert replaced.tenant_record_id == UUID7_TENANT + assert direct.tenant_record_id == UUID7_TENANT + + +@pytest.mark.parametrize( + ("field_name", "value", "message"), + [ + ( + "monitoring_plan_reference", + "selection_monitoring_plan:Quarterly-Plan", + "opaque selection_monitoring_plan", + ), + ("job_profile_reference", "job_profile:RN-ICU", "opaque job_profile"), + ( + "selection_process_reference", + "selection_process:hiring-2026", + "opaque selection_process", + ), + ( + "protected_attribute_policy_reference", + "protected_attribute_policy:race-gender", + "opaque protected_attribute_policy", + ), + ("actor_reference", "actor:seonghobae", "opaque actor"), + ( + "reviewer_reference", + "actor:00000000-0000-0000-0000-000000000000", + "opaque actor", + ), + ( + "population_snapshot_reference", + "population_snapshot:FFFFFFFF-FFFF-FFFF-FFFF-FFFFFFFFFFFF", + "opaque population_snapshot", + ), + ], +) +def test_references_reject_value_bearing_sentinel_and_noncanonical_suffixes( + field_name: str, + value: object, + message: str, +) -> None: + """Reject unsafe reference suffixes through builder, replace, and direct construction.""" + with pytest.raises(ValueError, match=message): + _build(**{field_name: value}) + + packet = _build() + with pytest.raises(ValueError, match=message): + replace(packet, **{field_name: value}) + + with pytest.raises(ValueError, match=message): + _direct(**{field_name: value}) + + +@pytest.mark.parametrize( + ("field_name", "prefix"), + [ + ("monitoring_plan_reference", "selection_monitoring_plan"), + ("job_profile_reference", "job_profile"), + ("selection_process_reference", "selection_process"), + ("population_snapshot_reference", "population_snapshot"), + ("outcome_snapshot_reference", "selection_outcome_snapshot"), + ("protected_attribute_policy_reference", "protected_attribute_policy"), + ("small_sample_policy_reference", "small_sample_policy"), + ("statistical_plan_reference", "statistical_plan"), + ("actor_reference", "actor"), + ("reviewer_reference", "actor"), + ], +) +def test_uuid1_trust_reference_is_rejected_by_all_construction_paths( + field_name: str, + prefix: str, +) -> None: + """UUIDv1 timestamp/node metadata must never enter an aggregate trust-reference field.""" + value = f"{prefix}:{UUID1_ID}" + with pytest.raises(ValueError, match=field_name): + _build(**{field_name: value}) + + with pytest.raises(ValueError, match=field_name): + replace(_build(), **{field_name: value}) + + with pytest.raises(ValueError, match=field_name): + _direct(**{field_name: value}) diff --git a/packages/selection-monitoring/tests/test_string_runtime_evidence_integrity.py b/packages/selection-monitoring/tests/test_string_runtime_evidence_integrity.py new file mode 100644 index 000000000..07fdda7ab --- /dev/null +++ b/packages/selection-monitoring/tests/test_string_runtime_evidence_integrity.py @@ -0,0 +1,131 @@ +"""Regression coverage for string-subclass evidence-boundary integrity.""" + +from __future__ import annotations + +from datetime import date, datetime, timezone + +import pytest + +from orgmetra_selection_monitoring import build_selection_outcome_monitoring_plan + + +class ForgedReference(str): + """String subclass that forges namespace and UUID suffix validation.""" + + def startswith(self, prefix, *args): # type: ignore[no-untyped-def] + """Pretend the hostile value carries every requested namespace.""" + return True + + def split(self, sep=None, maxsplit=-1): # type: ignore[no-untyped-def] + """Feed validation a canonical UUIDv4 suffix instead of stored text.""" + return ["evil", "11111111-1111-4111-8111-111111111111"] + + +class ForgedTenantUUIDText(str): + """String subclass that forges UUID parsing and canonical-equality checks.""" + + def replace(self, old, new, *args): # type: ignore[no-untyped-def] + """Feed UUID() canonical text instead of the stored hostile tenant text.""" + canonical = "12345678-1234-4234-8234-123456789abc" + return canonical.replace(old, new, *args) + + def __eq__(self, other): # type: ignore[no-untyped-def] + """Claim canonical equality while retaining the hostile underlying text.""" + if other is None: + return False + return True + + def __ne__(self, other): # type: ignore[no-untyped-def] + """Keep UUID constructor sentinel checks working while defeating canonicality.""" + if other is None: + return True + return False + + +class ForgedGovernanceCode(str): + """String subclass that can satisfy closed-code comparisons with hostile text.""" + + def __eq__(self, other): # type: ignore[no-untyped-def] + return True + + def __ne__(self, other): # type: ignore[no-untyped-def] + return False + + def __hash__(self) -> int: + return hash("quarterly_selection_governance") + + +class DigestSubclass(str): + """Semantically valid-looking digest carried by an untrusted runtime subclass.""" + + +def valid_kwargs() -> dict[str, object]: + """Return one otherwise valid monitoring-plan input.""" + return { + "tenant_record_id": "11111111-1111-4111-8111-111111111111", + "monitoring_plan_reference": "selection_monitoring_plan:10000000-0000-4000-8000-000000000001", + "job_profile_reference": "job_profile:10000000-0000-4000-8000-000000000002", + "selection_process_reference": "selection_process:10000000-0000-4000-8000-000000000003", + "population_snapshot_reference": "population_snapshot:10000000-0000-4000-8000-000000000004", + "population_snapshot_digest": "a" * 64, + "outcome_snapshot_reference": "selection_outcome_snapshot:10000000-0000-4000-8000-000000000005", + "outcome_snapshot_digest": "b" * 64, + "protected_attribute_policy_reference": "protected_attribute_policy:10000000-0000-4000-8000-000000000006", + "protected_attribute_policy_digest": "c" * 64, + "small_sample_policy_reference": "small_sample_policy:10000000-0000-4000-8000-000000000007", + "small_sample_policy_digest": "d" * 64, + "statistical_plan_reference": "statistical_plan:10000000-0000-4000-8000-000000000008", + "statistical_plan_digest": "e" * 64, + "actor_reference": "actor:10000000-0000-4000-8000-000000000009", + "reviewer_reference": "actor:10000000-0000-4000-8000-00000000000a", + "monitoring_start": date(2026, 1, 1), + "monitoring_end": date(2026, 3, 31), + "purpose_code": "selection_outcome_monitoring", + "reason_code": "quarterly_selection_governance", + "generated_at": datetime(2026, 4, 2, 8, 30, tzinfo=timezone.utc), + } + + +def test_rejects_reference_string_subclass_that_can_forge_namespace_validation() -> None: + """Canonical evidence must not retain text that only pretended to match a namespace.""" + kwargs = valid_kwargs() + kwargs["monitoring_plan_reference"] = ForgedReference("evil:payload") + + with pytest.raises(ValueError, match="monitoring_plan_reference"): + build_selection_outcome_monitoring_plan(**kwargs) + + +def test_rejects_tenant_string_subclass_that_can_forge_uuid_validation() -> None: + """Authoritative tenant identity must be exact built-in text before UUID parsing.""" + kwargs = valid_kwargs() + kwargs["tenant_record_id"] = ForgedTenantUUIDText("not-a-tenant-uuid") + + with pytest.raises(ValueError, match="tenant_record_id"): + build_selection_outcome_monitoring_plan(**kwargs) + + +def test_rejects_purpose_code_string_subclass_that_can_forge_closed_code_check() -> None: + """Purpose evidence must be exact built-in text before fixed-code comparison.""" + kwargs = valid_kwargs() + kwargs["purpose_code"] = ForgedGovernanceCode("attacker_controlled_purpose") + + with pytest.raises(ValueError, match="purpose_code"): + build_selection_outcome_monitoring_plan(**kwargs) + + +def test_rejects_reason_code_string_subclass_that_can_forge_closed_code_membership() -> None: + """Reason evidence must be exact built-in text before allow-list membership.""" + kwargs = valid_kwargs() + kwargs["reason_code"] = ForgedGovernanceCode("attacker_controlled_reason") + + with pytest.raises(ValueError, match="reason_code"): + build_selection_outcome_monitoring_plan(**kwargs) + + +def test_rejects_digest_string_subclass_before_canonical_evidence_binding() -> None: + """Digest evidence must use the same exact built-in string boundary as other trust text.""" + kwargs = valid_kwargs() + kwargs["population_snapshot_digest"] = DigestSubclass("a" * 64) + + with pytest.raises(ValueError, match="population_snapshot_digest"): + build_selection_outcome_monitoring_plan(**kwargs) diff --git a/packages/selection-monitoring/tests/test_temporal_evidence_integrity.py b/packages/selection-monitoring/tests/test_temporal_evidence_integrity.py new file mode 100644 index 000000000..236643080 --- /dev/null +++ b/packages/selection-monitoring/tests/test_temporal_evidence_integrity.py @@ -0,0 +1,162 @@ +"""Regression tests for exact temporal types in immutable monitoring evidence.""" + +from __future__ import annotations + +from datetime import date, datetime, timedelta, timezone, tzinfo + +import pytest + +from orgmetra_selection_monitoring import build_selection_outcome_monitoring_plan + + +class ForgedDate(date): + """Date subclass able to forge the canonical evidence rendering.""" + + def isoformat(self) -> str: + """Return a date different from the underlying business date.""" + return "2099-12-31" + + +class ForgedDateTime(datetime): + """Datetime subclass able to forge the canonical evidence rendering.""" + + def astimezone(self, tz=None): # type: ignore[no-untyped-def] + """Keep the subclass alive across the UTC normalization call.""" + return self + + def isoformat(self, *args, **kwargs) -> str: # type: ignore[no-untyped-def] + """Return an instant different from the underlying evidence instant.""" + return "2099-12-31T23:59:59+00:00" + + +class MutableTimezone(tzinfo): + """Timezone provider whose offset can change after packet issuance.""" + + def __init__(self, offset: timedelta) -> None: + """Store one caller-controlled offset.""" + self.offset = offset + + def utcoffset(self, dt: datetime | None) -> timedelta: + """Return the current mutable offset.""" + return self.offset + + def dst(self, dt: datetime | None) -> timedelta: + """Expose no daylight-saving adjustment.""" + return timedelta(0) + + def tzname(self, dt: datetime | None) -> str: + """Return a stable diagnostic timezone name.""" + return "MutableTimezone" + + +class RaisingTimezone(tzinfo): + """Timezone provider that raises while resolving its UTC offset.""" + + def utcoffset(self, dt: datetime | None) -> timedelta: + """Raise caller-controlled behavior at the trust boundary.""" + raise RuntimeError("provider failure") + + def dst(self, dt: datetime | None) -> timedelta: + """Expose no daylight-saving adjustment.""" + return timedelta(0) + + +def valid_kwargs() -> dict[str, object]: + """Return one otherwise valid aggregate-monitoring plan input.""" + return { + "tenant_record_id": "11111111-1111-4111-8111-111111111111", + "monitoring_plan_reference": "selection_monitoring_plan:10000000-0000-4000-8000-000000000001", + "job_profile_reference": "job_profile:10000000-0000-4000-8000-000000000002", + "selection_process_reference": "selection_process:10000000-0000-4000-8000-000000000003", + "population_snapshot_reference": "population_snapshot:10000000-0000-4000-8000-000000000004", + "population_snapshot_digest": "a" * 64, + "outcome_snapshot_reference": "selection_outcome_snapshot:10000000-0000-4000-8000-000000000005", + "outcome_snapshot_digest": "b" * 64, + "protected_attribute_policy_reference": "protected_attribute_policy:10000000-0000-4000-8000-000000000006", + "protected_attribute_policy_digest": "c" * 64, + "small_sample_policy_reference": "small_sample_policy:10000000-0000-4000-8000-000000000007", + "small_sample_policy_digest": "d" * 64, + "statistical_plan_reference": "statistical_plan:10000000-0000-4000-8000-000000000008", + "statistical_plan_digest": "e" * 64, + "actor_reference": "actor:10000000-0000-4000-8000-000000000009", + "reviewer_reference": "actor:10000000-0000-4000-8000-00000000000a", + "monitoring_start": date(2026, 1, 1), + "monitoring_end": date(2026, 3, 31), + "purpose_code": "selection_outcome_monitoring", + "reason_code": "quarterly_selection_governance", + "generated_at": datetime(2026, 4, 2, 8, 30, tzinfo=timezone.utc), + } + + +@pytest.mark.parametrize("field_name", ["monitoring_start", "monitoring_end"]) +def test_rejects_date_subclasses_that_can_forge_canonical_business_time(field_name: str) -> None: + """Do not let subclass methods rewrite immutable business-time evidence.""" + kwargs = valid_kwargs() + kwargs[field_name] = ForgedDate(2026, 1, 1 if field_name == "monitoring_start" else 3) + + with pytest.raises(ValueError, match="calendar date"): + build_selection_outcome_monitoring_plan(**kwargs) + + +def test_rejects_datetime_subclasses_that_can_forge_canonical_recorded_time() -> None: + """Do not let subclass methods rewrite immutable recorded-time evidence.""" + kwargs = valid_kwargs() + kwargs["generated_at"] = ForgedDateTime(2026, 4, 2, 8, 30, tzinfo=timezone.utc) + + with pytest.raises(ValueError, match="timezone-aware"): + build_selection_outcome_monitoring_plan(**kwargs) + + +def test_detaches_mutable_timezone_from_immutable_generated_time() -> None: + """Do not let a timezone provider rewrite canonical evidence after issuance.""" + provider = MutableTimezone(timedelta(hours=9)) + kwargs = valid_kwargs() + kwargs["generated_at"] = datetime(2026, 4, 2, 17, 30, tzinfo=provider) + + plan = build_selection_outcome_monitoring_plan(**kwargs) + before = plan.canonical_json() + provider.offset = timedelta(hours=-7) + + assert plan.canonical_json() == before + assert plan.generated_at == datetime(2026, 4, 2, 8, 30, tzinfo=timezone.utc) + assert plan.generated_at.tzinfo is timezone.utc + + +def test_rejects_future_generated_time() -> None: + """Do not seal a monitoring plan for a system time that has not occurred.""" + kwargs = valid_kwargs() + kwargs["generated_at"] = datetime(2099, 1, 1, tzinfo=timezone.utc) + + with pytest.raises(ValueError, match="generated_at must not be in the future"): + build_selection_outcome_monitoring_plan(**kwargs) + + +def test_normalizes_timezone_provider_failure() -> None: + """Do not leak arbitrary timezone-provider exceptions across the evidence boundary.""" + kwargs = valid_kwargs() + kwargs["generated_at"] = datetime(2026, 4, 2, 8, 30, tzinfo=RaisingTimezone()) + + with pytest.raises(ValueError, match="timezone-aware"): + build_selection_outcome_monitoring_plan(**kwargs) + + +def test_rejects_timezone_normalization_overflow() -> None: + """Fail closed when a valid offset cannot be represented as a UTC datetime.""" + kwargs = valid_kwargs() + kwargs["generated_at"] = datetime.min.replace(tzinfo=timezone(timedelta(hours=14))) + + with pytest.raises(ValueError, match="timezone-aware"): + build_selection_outcome_monitoring_plan(**kwargs) + + +def test_rejects_post_construction_timezone_reinjection() -> None: + """Do not emit evidence after low-level replacement of the frozen UTC instant.""" + plan = build_selection_outcome_monitoring_plan(**valid_kwargs()) + object.__setattr__( + plan, + "generated_at", + datetime(2026, 4, 2, 17, 30, tzinfo=timezone(timedelta(hours=9))), + ) + + with pytest.raises(ValueError, match="timezone-aware"): + plan.canonical_json() diff --git a/tests/test_foundation_ci_dependency_hygiene.sh b/tests/test_foundation_ci_dependency_hygiene.sh old mode 100644 new mode 100755 index 2c0f5087f..c069a7958 --- a/tests/test_foundation_ci_dependency_hygiene.sh +++ b/tests/test_foundation_ci_dependency_hygiene.sh @@ -90,3 +90,18 @@ for package_name in coverage iniconfig packaging pluggy Pygments pytest pytest-c exit 1 fi done + +shopt -s nullglob +delegated_artifact_contracts=( + "${repository_root}"/tests/test_foundation_ci_*_artifact.sh +) +shopt -u nullglob + +if [[ "${#delegated_artifact_contracts[@]}" -eq 0 ]]; then + printf 'Foundation CI must own at least one delegated artifact contract.\n' >&2 + exit 1 +fi + +for artifact_contract in "${delegated_artifact_contracts[@]}"; do + bash "${artifact_contract}" +done diff --git a/tests/test_foundation_ci_selection_monitoring_artifact.sh b/tests/test_foundation_ci_selection_monitoring_artifact.sh new file mode 100755 index 000000000..a2ae1a8c8 --- /dev/null +++ b/tests/test_foundation_ci_selection_monitoring_artifact.sh @@ -0,0 +1,30 @@ +#!/usr/bin/env bash +set -euo pipefail + +repository_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +package_root="${repository_root}/packages/selection-monitoring" +requirements_path="${repository_root}/.github/requirements/foundation-test.txt" +retired_workflow="${repository_root}/.github/workflows/selection-monitoring-quality.yml" +venv_dir="/tmp/orgmetra-selection-monitoring-venv" + +if [[ -e "${retired_workflow}" ]]; then + printf 'Retired Selection Monitoring leaf workflow must not exist.\n' >&2 + exit 1 +fi + +rm -rf "${venv_dir}" +cleanup() { + rm -rf "${venv_dir}" +} +trap cleanup EXIT + +python -m venv "${venv_dir}" +"${venv_dir}/bin/python" -m pip install --require-hashes --no-deps --only-binary=:all: -r "${requirements_path}" +"${venv_dir}/bin/python" -m pip check +"${venv_dir}/bin/python" -m compileall -q "${package_root}/src" "${package_root}/tests" + +PYTHONPATH="${package_root}/src" \ +COVERAGE_FILE=/tmp/orgmetra-selection-monitoring.coverage \ + "${venv_dir}/bin/python" -m pytest \ + -c "${package_root}/pyproject.toml" \ + "${package_root}/tests"