diff --git a/docs/ASVS-L2-PHASE0-CHANGES.md b/docs/ASVS-L2-PHASE0-CHANGES.md index 6cce4acf..43b78ddc 100644 --- a/docs/ASVS-L2-PHASE0-CHANGES.md +++ b/docs/ASVS-L2-PHASE0-CHANGES.md @@ -328,7 +328,7 @@ tables in [`../CONNECTIONS.md`](../CONNECTIONS.md) §"Resource management & limi | Store — SQL Server | outbound | TDS over ODBC Driver 18, **default port 1433** | `encrypt` default true, `trust_server_certificate` default false; optional `ssl_root_cert` pins the server certificate via the Driver-18.1 `ServerCertificate` keyword; optional `multi_subnet_failover` for an AOAG listener | SQL / Integrated / Entra, with an opt-in `require_managed_identity` precondition (default off) | **yes** — `[store].server` | `[store].backend = sqlserver`, `[store].server`, `[store].port`, `[store].auth`, `[store].pool_size`, `[store].connect_timeout`, `[store].command_timeout`, `[store].warm_pool` | | Store — Postgres (asyncpg) | outbound | TCP; the model's `port` default is 1433 and a **left-at-default value is remapped to Postgres's conventional 5432** | TLS built from `encrypt` / `trust_server_certificate`; `ssl_root_cert` loads `ssl.create_default_context(cafile=…)`, so chain **and** hostname are still verified | username + password only — Postgres has **no managed-identity mode**, so it cannot satisfy `require_managed_identity` | **yes** — `[store].server` | `[store].backend = postgres`, plus the same `[store]` connection keys as SQL Server | | Active Directory — login binds | outbound | LDAPS (TLS) by default; a plain `ldap://` bind requires the explicit `ad_allow_insecure_ldap` opt-in | `ad_tls_verify` default true (`CERT_REQUIRED`); `ad_tls_verify=false` now **refuses at construction** without `MEFOR_ALLOW_INSECURE_TLS`; `ad_tls_ca_cert_file` anchors an internal CA without disabling verification | a service-account simple bind (`ad_bind_dn` + `ad_bind_password` / `ad_bind_password_secret`), then a **second bind as the user**, then 1–2 SUBTREE searches | **yes** — `[auth].ad_server` | `[auth].ad_server`, `ad_domain`, `ad_bind_dn`, `ad_user_search_base`, `ad_group_search_base`, `ad_use_nested_groups`, `ad_connect_timeout`, `ad_receive_timeout` | -| Active Directory — directory session reconciler (ADR 0079) | outbound (periodic) | the same LDAPS hop, off the login path | as above | the service-account bind only — one `resolve_principal` bind per signed-in directory user per pass | same server | `[auth].ad_session_recheck_seconds` (0 = OFF, floored at 60 s), `ad_session_recheck_max_users`, `ad_session_recheck_strikes`, `ad_session_revoke_max`, `ad_session_revoke_max_fraction` | +| Active Directory — directory session reconciler (ADR 0079) | outbound (periodic) | the same LDAPS hop, off the login path | as above | the service-account bind only — one `resolve_principal` bind per signed-in directory user per pass | same server | `[auth].ad_session_recheck_seconds` (default 300 s, floored at 60 s; 0 = OFF), `ad_session_recheck_max_users`, `ad_session_recheck_strikes`, `ad_session_revoke_max`, `ad_session_revoke_max_fraction` | | Kerberos / SPNEGO browser SSO | n/a — **the engine opens no socket** | one SPNEGO *server* step against the host's SSPI (Windows) or GSSAPI/krb5 (Linux) provider using the local keytab / machine credential; any KDC traffic belongs to the OS provider | n/a — the engine terminates nothing here | the browser's SPNEGO ticket, validated by the OS provider | no — the host's realm / keytab | `[auth].kerberos_enabled` (experimental, off by default), `[auth].kerberos_spn` | | OIDC IdP — token endpoint (ADR 0142) | outbound | HTTPS POST to the operator-**pinned** endpoint; port from the URL. There is **no `.well-known` discovery** — every endpoint is pinned, so no attacker-influenced URL exists | `ssl.create_default_context(cafile=oidc_tls_ca_cert_file)` or the OS store, **always** `check_hostname=True` + `CERT_REQUIRED` (no verify-off escape by design); redirects are never followed | the confidential-client secret plus the authorization code and the PKCE verifier | **yes** — `[auth].oidc_token_endpoint` | `[auth].oidc_token_endpoint`, `oidc_client_id`, `oidc_client_secret` (env `MEFOR_AUTH_OIDC_CLIENT_SECRET`) or `oidc_client_secret_ref`, `oidc_allowed_endpoints`, `oidc_tls_ca_cert_file` | | OIDC IdP — authorization endpoint + authorization-code **callback** (ADR 0142) | **n/a — the engine opens no socket on this leg**: the BROWSER is redirected to the operator-pinned authorization endpoint, and the IdP returns the code to the engine's own `[api]` listener | HTTPS redirect to `[auth].oidc_authorization_endpoint` (port from the URL); the code arrives back on `[auth].oidc_redirect_path` (default `/ui/oidc/callback`, fixed in this release), whose absolute `redirect_uri` is derived from `[security].web_console_public_address` | the browser↔IdP leg carries the browser's own TLS, not the engine's; the callback leg rides the §5.2 API listener's posture. The endpoint is host-gated by `oidc_allowed_endpoints` | server-side PKCE `code_verifier` + `state` (constant-time compare) + `nonce` held in the bounded pending-flow cache — only the opaque `flow_id` ever reaches the browser, in a `__Host-`-prefixed cookie the callback requires | **yes** — `[auth].oidc_authorization_endpoint` is the operator-supplied external location, and `oidc_redirect_path` is the **callback URL** the requirement names by example | `[auth].oidc_authorization_endpoint`, `oidc_redirect_path`, `oidc_allowed_endpoints`, `oidc_flow_ttl_seconds`, `oidc_flow_cache_max`, `[security].web_console_public_address` | diff --git a/docs/CONFIGURATION.md b/docs/CONFIGURATION.md index 77b28fbe..427b6f10 100644 --- a/docs/CONFIGURATION.md +++ b/docs/CONFIGURATION.md @@ -63,7 +63,7 @@ backend-limited. | `fifo_claim_prepared` | bool | `false` | **SQL Server only** (ADR 0114 sub-lever B). Stabilizes the pooled claim's statement text (one JSON lanes parameter) and retains a prepared claim cursor on store-owned dedicated connections (INGRESS/ROUTED; the non-DDL fallback lane to `fifo_claim_proc`). **Logs + no-ops unless `fifo_claim_fold_reset` is on** (without the fold the finally-guard's reset would evict the one-slot prepare cache every call). `false` = byte-identical. Flip only after its own §8 gate (AC-14). | | `encryption_key` | secret | — | **env only** (`MEFOR_STORE_ENCRYPTION_KEY`); base64 32-byte **active** key — when set, PHI columns (`raw`/`payload` + `error`/`last_error`/`detail`) are AES-256-GCM-encrypted at rest. Mint one with `messagefoundry gen-key`. Empty = off. See [PHI.md §3](PHI.md#3-encryption-at-rest). | | `encryption_keys_retired` | secret | — | **env only** (`MEFOR_STORE_ENCRYPTION_KEYS_RETIRED`); comma-separated base64 **decrypt-only** keys kept available during a rotation until `messagefoundry rotate-key` finishes re-encrypting under the active key (ASVS 11.2.2). | -| `aad_bind` | bool | `false` | **not a secret** (`MEFOR_STORE_AAD_BIND`); the hardened **Posture-B** setting (ASVS 11.3.3, [ADR 0019](adr/0019-pluggable-keyprovider-hsm-kms-vault.md)). When `true`, new at-rest AES-256-GCM writes use the cell-bound `mfenc:v2` writer — each value is bound to its `(table, column, row)` cell via GCM Associated Data, so a ciphertext cut-and-pasted into another cell **fails the auth tag** (dead-lettered `CipherError`) instead of silently decrypting. Off by default → the frozen `mfenc:v1` writer (byte-identical at rest). No effect without an `encryption_key` (the identity cipher has nothing to bind). Legacy `v1` rows still decrypt (dual-read); `messagefoundry rotate-key` upgrades them `v1`→`v2`. | +| `aad_bind` | bool | `true` | **not a secret** (`MEFOR_STORE_AAD_BIND`); cell binding (ASVS 11.3.3, [ADR 0019](adr/0019-pluggable-keyprovider-hsm-kms-vault.md)). New at-rest AES-256-GCM writes use the cell-bound `mfenc:v2` writer — each value is bound to its `(table, column, row)` cell via GCM Associated Data, so a ciphertext cut-and-pasted into another cell **fails the auth tag** (dead-lettered `CipherError`) instead of silently decrypting. **On by default** (ADR 0148 GIVEN 1: the shipped configuration runs the hardened path). Setting it `false` selects the frozen `mfenc:v1` writer (byte-identical at rest) and is a **loosening** — `security_loosenings()` names it, so the opt-out is never silent. No effect without an `encryption_key` (the identity cipher has nothing to bind). Legacy `v1` rows still decrypt (dual-read); `messagefoundry rotate-key` upgrades them `v1`→`v2`, so the default is safe and reversible on an existing store. | | `key_provider` | enum | `auto` | selects **how** the active/retired DEK bytes are *sourced* — never how they are used (the cipher, keyring, and `mfenc:v1` format are unchanged; ADR 0019, ASVS 13.3.3). `auto` (default) is the env-then-DPAPI ladder, **byte-identical** to the pre-seam behavior; `env`/`dpapi` pin a single built-in source; `aws_kms`·`azure_kv`·`gcp_kms`·`vault`·`pkcs11` envelope-decrypt a wrapped DEK inside an HSM/KMS/Vault (lazy **optional extras — not built yet**; selecting one **fails closed** at `serve`, never a silent downgrade). Names a *provider*, not key material, so it is **not** a secret. | | `require_encryption` | bool | `false` | when `true`, `serve` **refuses to start** without an encryption key in **any** environment, even a synthetic one. Off by default. | | `allow_unencrypted_phi` | | | **→ moved to `[security].allow_unencrypted_phi`** (ADR 0118) — set it there; no longer accepted in `[store]`. | @@ -456,7 +456,7 @@ including the surfaces that remain **unbounded** at this release — is | `ad_allow_insecure_ldap` | bool | `false` | explicit opt-in to a non-`ldaps://` bind (trusted-network dev only) | | `ad_connect_timeout` | float | `10.0` | seconds — bounds the LDAP/LDAPS **TCP connect** on every `ldap3` `Server` the authenticator builds (ASVS 13.1.3). Must be finite and `> 0`; `0`, negative, `inf` and `NaN` are refused at config load. `ldap3`'s own default is `None` (wait forever), so without this an unresponsive DC pinned a thread-pool worker indefinitely | | `ad_receive_timeout` | float | `10.0` | seconds — bounds **each LDAP response read** (both binds and every search) on every `ldap3` `Connection`. Same finite-positive validation | -| `ad_session_recheck_seconds` | int | `0` | **Directory session reconciliation** ([ADR 0079](adr/0079-kerberos-idp-session-coordination.md) mechanism 2). How often to re-resolve directory principals holding **live** sessions and revoke those AD has disabled or deleted — without it, an AD disable does not take effect until the `[security].max_session_hours` cap (12 h). **`0` = OFF, the default** (no task, byte-identical upgrade); a non-zero value is floored at **60 s** (a pass costs one LDAP bind per signed-in directory user). Requires `ad_enabled` — refused otherwise rather than silently dead. **Recommended `300` for an off-loopback PHI deployment.** | +| `ad_session_recheck_seconds` | int | `300` | **Directory session reconciliation** ([ADR 0079](adr/0079-kerberos-idp-session-coordination.md) mechanism 2). How often to re-resolve directory principals holding **live** sessions and revoke those AD has disabled or deleted — without it, an AD disable does not take effect until the `[security].max_session_hours` cap (12 h). **`300` (five minutes) is the default** (ADR 0148 GIVEN 1 — the hardened path is the shipped path), floored at **60 s** (a pass costs one LDAP bind per signed-in directory user). `0` disables the loop and is a **loosening** once AD is on — `security_loosenings()` names it. The default is **inert without AD** (`should_reconcile()` also needs an LDAP client), so a non-AD deployment is unaffected; an **explicit** non-zero value without `ad_enabled` is still refused rather than left silently dead. | | `ad_session_recheck_strikes` | int | `2` | Consecutive passes a principal must fail to resolve before its sessions are revoked. The directory lookup collapses *disabled*, *deleted* and *the search matched nothing* into one answer, so a single ambiguous result must never revoke. Range 1–10. | | `ad_session_recheck_max_users` | int | `200` | Per-pass bind budget. Beyond this, remaining users are picked up by later passes (least-recently-probed first), so a large estate degrades to a longer effective interval instead of a bind storm. | | `ad_session_revoke_max` | int | `5` | **Mass-revoke circuit breaker**, absolute half. A bad search base / moved OU / service account that lost read rights answers "not found" for *every* user — indistinguishable from "everyone was disabled". | @@ -525,7 +525,7 @@ the rest are forward-compat placeholders for the future engine broker (accepted- | `forward_tls_verify` | bool | `true` | verify + hostname-check the collector's certificate. `false` is the documented **insecure** opt-out (`CERT_NONE`, no CA file needed) — lab / pinned-network only | | `forward_tls_client_cert` | str | — | optional PEM cert+key chain for **mutual** TLS to the collector | | `forward_hop_attested` | bool | `false` | **acknowledged opt-out** for a plaintext / unverified-TLS collector hop (#200, ADR 0092 — the `[logging]` sibling of a connection's `tls_hop_attested`). A hop that is not verified TLS is now decided by the shared posture gradient: **refused** on an enforcing PHI instance, warned on a non-enforcing PHI instance, allowed for loopback / synthetic. Set this (with a reason) to affirm the hop is secure by other means — e.g. a dedicated out-of-band management VLAN | -| `forward_hop_attested_reason` | str | — | why the hop is secure, recorded for the audit trail. Only valid **with** `forward_hop_attested = true`, and must be non-empty | +| `forward_hop_attested_reason` | str | — | why the hop is secure, recorded for the audit trail. **Mandatory when `forward_hop_attested = true`** (ADR 0153 retro-fitted the flag-implies-reason rule: an attestation that suppresses a refusal must record WHY, or it is worthless when audited) — the flag alone now fails at load. Rejected without the flag, and must be non-empty | | `require_time_sync` | bool | `false` | **opt-in** startup clock-sync gate (ASVS 16.2.2, ADR 0080): before listeners start, probe `ntp_peer` and warn on skew. Requires `ntp_peer`. Default = no-op | | `ntp_peer` | str | — | NTP/SNTP host to compare the local clock against (**required** when `require_time_sync`) | | `time_sync_max_skew_seconds` | float | `2.0` | \|local − peer\| above this is "skewed" (must be > 0) | @@ -700,15 +700,20 @@ checked against the resolved (`env()`-substituted) destination. > The webhook/SMTP **alert** sinks carry no message bodies (no PHI) and keep their own host allowlists > in `[alerts]` (`webhook_allowed_hosts` / `smtp_allowed_hosts`). -> **Cleartext (`http://`) egress is refused only on a PHI posture (ASVS 12.2.1).** Separately from -> this allowlist, a plaintext `http://` outbound to a **non-loopback** host is decided by the instance -> PHI posture in [`config/tls_policy.py`](../messagefoundry/config/tls_policy.py) -> (`insecure_hop_disposition`), enforced at construction by `refuse_cleartext_egress` -> ([`transports/rest.py`](../messagefoundry/transports/rest.py)): a loopback / on-box, per-hop-attested, -> or **non-PHI (synthetic)** hop is **allowed**; a **production PHI** hop with no attestation is -> **refused** (fail-closed); a non-production PHI hop refuses unless the clamped audited escape -> downgrades it to a warning. A non-PHI instance therefore keeps cleartext http egress **by design** -> (ADR 0115 forbids unconditionally refusing it), so 12.2.1 stays Partial on a non-PHI Posture-A box. +> **Cleartext (`http://`) egress is refused, whatever the data label (ASVS 12.2.1, +> [ADR 0153](adr/0153-collapse-the-posture-gradient-no-data-label-may-allow-a-cleartext-hop.md)).** +> Separately from this allowlist, a plaintext `http://` outbound to a **non-loopback** host is decided by +> [`config/tls_policy.py`](../messagefoundry/config/tls_policy.py)'s `insecure_hop_disposition`, enforced +> at construction by `refuse_cleartext_egress` +> ([`transports/rest.py`](../messagefoundry/transports/rest.py)). The precedence is: a loopback / on-box +> hop is **allowed**; a per-connection `tls_hop_attested` hop is **allowed** (the operator asserts it is +> secure by other means); a per-connection `cleartext_accepted` hop is **crossed with a loud, audited +> WARN** (the operator accepts that it is not secure); a non-enforcing instance +> (`[security].enforcement = warn`) **warns**; everything else is **refused** (fail-closed). +> **`data_class` is no longer read here** — a `synthetic` label used to allow every cleartext hop +> silently, which made a typo in one file indistinguishable from a deliberate declaration, with every +> transport hop in the product as its blast radius. `MEFOR_ALLOW_INSECURE_TLS` no longer influences this +> decision either; it survives for the non-connection cells that have nowhere to carry a declaration. ### `[shadow]` Parallel-run / **shadow-instance** egress suppression (#15). A *shadow* MessageFoundry processes real diff --git a/docs/CONNECTIONS.md b/docs/CONNECTIONS.md index 88a59deb..76c1b4aa 100644 --- a/docs/CONNECTIONS.md +++ b/docs/CONNECTIONS.md @@ -1000,7 +1000,7 @@ report, plain text); this connector delivers it to `host:port` from `sender` to | `subject` | str / `env()` | `""` | Static subject (a per-message subject is a Phase-2 follow-up). | | `username` | str / `env()` / None | `None` | SMTP `AUTH` user — put the secret in `env()`. | | `password` | str / `env()` / None | `None` | SMTP `AUTH` password — `env()` only. AUTH is sent **over TLS only**; a cleartext-credential config is refused. | -| `use_tls` | bool | `True` | STARTTLS by default. `False` puts the message **body** (PHI) on the wire in the clear, so it is doubly gated: it needs the explicit opt-in (`MEFOR_ALLOW_INSECURE_TLS`, now read through the **clamped** check — inert on an enforcing production-PHI instance — or the connection's `tls_hop_attested`), **and** the hop itself goes through the shared posture gradient (#200, ADR 0092): refused on an enforcing PHI instance, warned on a non-enforcing PHI one, allowed for loopback / synthetic / attested. Matches the raw-TCP / X12 / plaintext-DICOM / anonymous-FTP cleartext egress paths. | +| `use_tls` | bool | `True` | STARTTLS by default. `False` puts the message **body** (PHI) on the wire in the clear, so it is doubly gated: it needs an explicit opt-in (`MEFOR_ALLOW_INSECURE_TLS` read through the **clamped** check, the connection's `tls_hop_attested`, or its `cleartext_accepted` declaration), **and** the hop itself goes through the shared authority (#200, ADR 0092 as amended by ADR 0153): loopback and attested hops ALLOW, a `cleartext_accepted` hop WARNs + audits, a non-enforcing instance WARNs, everything else REFUSES — **no data label relaxes it**. SMTP AUTH over cleartext stays refused OUTRIGHT, by any route. Matches the raw-TCP / X12 / plaintext-DICOM / anonymous-FTP cleartext egress paths. | | `timeout_seconds` | float | `30.0` | | | `encoding` | str | `"utf-8"` | | @@ -1370,6 +1370,79 @@ outbound("MLLP-OUT_PAYER_ELIG", MLLP(host=env("payer_host"), port=2575, reingres # a Handler Sends the eligibility query to MLLP-OUT_PAYER_ELIG; its reply re-ingresses into IB-LOOP_PAYER_ELIG. ``` +## Declaring a cleartext hop (`cleartext_accepted`) + +An **outbound** connection whose hop has no TLS is **refused** at `messagefoundry check` / dry-run / +reload / the serve pre-flight under the default `[security].enforcement = enforce` +([ADR 0153](adr/0153-collapse-the-posture-gradient-no-data-label-may-allow-a-cleartext-hop.md)). No data +label relaxes that — `data_class = "synthetic"` used to allow every cleartext hop silently, and no longer +does. There are exactly three ways such a hop crosses: + +| | claim | disposition | +|---|---|---| +| the hop is **on-box** (loopback / `localhost` / empty host) | not a network exposure | ALLOW | +| `cleartext_accepted = true` + `cleartext_reason` | this hop is **not** secure, and we accept that | **WARN** — crossed, loudly logged **and recorded at every construction** | +| `[security].enforcement = warn` | the instance-wide refuse/warn dial is at `warn` | WARN — but **only for the raw transports** (`MLLP()`, `Tcp()`, `X12()`, `DICOM()`, `Email()`, `Ftp()`). The HTTP family (`Rest()`, `Soap()`, `FHIR()`, `DICOMweb()`, `FhirLookup()`) shipped these refusals unconditionally, and ADR 0092 decision 5 forbids a cell getting weaker, so a no-loosen floor turns that WARN back into a REFUSE there. The dial is **not** a substitute for the declaration | + +A fourth route exists in the engine but has **no authoring surface on a connection today**: +`tls_hop_attested` ("this hop *is* secure by means the engine cannot see" — proxy-terminated TLS, a +genuinely isolated segment) yields a silent ALLOW, but no transport factory takes it and it is not a +`connections.toml` key, so you cannot set it on an inbound or outbound. Do **not** reach for it; use +`cleartext_accepted`. (The `[logging].forward_hop_attested` sibling in `messagefoundry.toml` *is* +settable — see [CONFIGURATION.md](CONFIGURATION.md).) + +The two claims are deliberately **separate fields with opposite meanings**. Do not describe a peer that +simply cannot do TLS as attested: that writes a false statement into the one field that exists to be +trustworthy when it is audited, and it leaves the audit trail unable to tell a proxy-terminated hop from +plaintext on a flat network. + +Both surfaces accept the pair — code-first on `outbound(...)`, or as **top-level** keys in +`connections.toml` (they are governance declarations, not transport settings, so they do **not** go under +`[outbound.settings]`). A `FhirLookup(...)` read connection takes the same two keyword arguments: + +```python +outbound( + "OB_LEGACY_LAB", + Tcp(host=env("lab_host"), port=env("lab_port")), + cleartext_accepted=True, + cleartext_reason="vendor firmware predates TLS; segment is not isolated", +) +``` + +```toml +[[outbound]] +name = "OB_LEGACY_LAB" +transport = "tcp" +cleartext_accepted = true +cleartext_reason = "vendor firmware predates TLS; segment is not isolated" + [outbound.settings] + host = "10.4.2.15" + port = 5000 +``` + +| Key | Dir | Type | Default | Meaning | +|-----|-----|------|---------|---------| +| `cleartext_accepted` | out | bool | `false` | this outbound's hop is cleartext and that is accepted. Yields **WARN, never ALLOW** — the hop crosses, but every construction logs it and records a line naming the connection, the cell, the host and the reason. It does **not** reach a `verify_tls = false` hop (encrypted-but-unauthenticated, not cleartext — that keeps the clamped `MEFOR_ALLOW_INSECURE_TLS` escape) or an SMTP `AUTH` over cleartext (refused outright) | +| `cleartext_reason` | out | str | — | **mandatory** when the flag is set (and rejected without it). The engine checks a reason is present and non-blank; it cannot check that it is *true* — a placeholder is a review problem, not a load problem | + +**`Tcp()` and `X12()`: the declaration is permanent, not transitional.** Those connectors have **no TLS +support at all** — no `tls` parameter, no `ssl` import — so there is no `tls = true` for them to migrate +to. On every other **outbound** transport (`MLLP()`, `Rest()`, `Soap()`, `FHIR()`, `DICOMweb()`, +`DICOM()`, `Email()`, `Ftp()`, `FhirLookup()`) the declaration should be read as **naming work to be +done**, and removed when the peer gains TLS. Adding TLS to raw TCP and X12 is tracked as BACKLOG #311. + +Two caveats on that list. `Http()` is an **inbound listener only** — it binds rather than dials, so it +has no hop to declare; inbound binds are governed by `--allow-insecure-bind` and the four exposed-gates, +not by this declaration (ADR 0153 decision 2 is Destination-only). And on `Ftp()` the declaration reaches +the **anonymous** plain-ftp hop only: a *credentialed* plain-ftp connection is refused outright, because +the credential itself would cross in the clear. + +**It is never invisible.** A declared hop appears in `messagefoundry check` (a `cleartext-accepted` line +listing the **whole** accepted set, so a broad rollout is obvious in review), in the connector's +construction WARN + audit record, and in `GET /security/posture`'s loosening list — with a deviation +entry in [SECURITY-LOOSENING.md](SECURITY-LOOSENING.md). That visibility is the mitigation: nothing stops +an operator declaring it on every destination, and the engine does not try to. + ## Per-connection retention, document pruning & diagnostics overrides A connection may **override** several service-wide `[…]` defaults for just itself. Each is set the same diff --git a/docs/PHI.md b/docs/PHI.md index e0f0f2c6..08c5d525 100644 --- a/docs/PHI.md +++ b/docs/PHI.md @@ -146,12 +146,12 @@ store's cipher registry — derived from the `cell_aad(...)` call sites and each `_CIPHER_COLUMNS` / migration / rotation passes — so they cannot diverge. See [§3](#3-encryption-at-rest). -**Cell binding is off by default.** Every write site above passes a cell AAD, but the AAD is **ignored** -unless `[store].aad_bind = true` — the default (`false`) runs the frozen `mfenc:v1` writer, which binds -no associated data. The AAD tuples are documented because they are what *would* bind, and what *does* -bind under `aad_bind = true` (`mfenc:v2`) and under `[store].cipher_provider = "vault_transit"` -(`mfenc:v3`, where the AAD is forwarded to Transit unconditionally). See -[§3](#3-encryption-at-rest). +**Cell binding is ON by default** (`[store].aad_bind = true`, ADR 0148 GIVEN 1). Every write site +above passes a cell AAD and, on the shipped default, that AAD is **bound** — writes use the `mfenc:v2` +writer. Setting `aad_bind = false` selects the frozen `mfenc:v1` writer, which binds no associated data +(the AAD is then computed and ignored), and is a **declared loosening** that `security_loosenings()` +names. The AAD is bound unconditionally under `[store].cipher_provider = "vault_transit"` (`mfenc:v3`, +where it is forwarded to Transit). See [§3](#3-encryption-at-rest). **Body format is irrelevant to the at-rest tier — they all ride the same cipher.** The `raw`/`payload` rows above are payload-agnostic, so non-HL7 PHI bodies are stored through the **same encrypting store @@ -241,14 +241,16 @@ for defense-in-depth without swapping the `aiosqlite` connector. version-agnostic `mfenc:` prefix (so a v2 row is recognised as already-encrypted), and the rotation scan anchors on the cipher's active-format prefix through the key fingerprint (so a v2-active rotation matches v2 rows and terminates). - **Cell binding — `[store].aad_bind`, default OFF (ASVS 11.3.3, ADR 0019).** Every store write site - already passes `cell_aad(table, column, *pk)` (the tuples are documented per row in §2), but the - **frozen `mfenc:v1` writer passes no associated data, so with the default `aad_bind = false` the AAD - is computed and then ignored — at-rest values are NOT cell-bound.** Set `[store].aad_bind = true` and - new writes become **`mfenc:v2` with the cell AAD bound**: a ciphertext cut-and-pasted from one cell - into another fails the GCM tag (dead-lettered, never silently accepted). Legacy `v1` rows stay - readable (dual-read) and **`messagefoundry rotate-key` upgrades them v1→v2**. `aad_bind` has no - effect without an encryption key (the identity cipher has nothing to bind). + **Cell binding — `[store].aad_bind`, default ON (ASVS 11.3.3, ADR 0019 as amended by ADR 0148 + GIVEN 1).** Every store write site passes `cell_aad(table, column, *pk)` (the tuples are documented + per row in §2), and on the shipped default new writes are **`mfenc:v2` with the cell AAD bound**: a + ciphertext cut-and-pasted from one cell into another fails the GCM tag (dead-lettered, never silently + accepted). Setting `aad_bind = false` selects the **frozen `mfenc:v1` writer, which passes no + associated data — the AAD is then computed and ignored, and at-rest values are NOT cell-bound**; that + is a declared loosening, named by `security_loosenings()`. Legacy `v1` rows stay readable (dual-read) + and **`messagefoundry rotate-key` upgrades them v1→v2**, so the default is safe on an existing store + and reversible. `aad_bind` has no effect without an encryption key (the identity cipher has nothing + to bind). **A third at-rest tier ships — `[store].cipher_provider = "vault_transit"` (`mfenc:v3`, ADR 0138).** This does not merely source the key: it **replaces the cipher object** ([store/crypto_transit.py](../messagefoundry/store/crypto_transit.py)), so every encrypt/decrypt runs @@ -355,10 +357,10 @@ application log files (`[logging].log_dir`). - **Encryption**, stated per tier rather than as one blanket rule: - *Database cells and the `[store].uploads_dir` sidecars* — the store cipher (AES-256-GCM, or - Transit under `vault_transit`) with the per-cell AAD in §2, keyed by the store DEK — **bound only - under `[store].aad_bind = true` (`mfenc:v2`) or `cipher_provider = "vault_transit"` (`mfenc:v3`); - under the shipped default `aad_bind = false` the AAD is computed and then ignored by the frozen - `mfenc:v1` writer.** + Transit under `vault_transit`) with the per-cell AAD in §2, keyed by the store DEK — **bound on the + shipped default (`[store].aad_bind = true` → `mfenc:v2`) and unconditionally under + `cipher_provider = "vault_transit"` (`mfenc:v3`); an operator who sets `aad_bind = false` selects the + frozen `mfenc:v1` writer, and the AAD is then computed and ignored.** - *`.mfbak` archives* — **a separate streaming codec, NOT the store cipher** ([store/backup_codec.py](../messagefoundry/store/backup_codec.py), whose own docstring says the cipher *mechanism* is net-new): chunked AES-256-GCM under the store DEK resolved directly by @@ -436,7 +438,8 @@ application log files (`[logging].log_dir`). | `alert_instance.reason` | `safe_text(…)[:200]` | `GET /alerts/active` under **`monitoring:diagnose`** — again not a PHI permission | same window, **RESOLVED instances only** — an open or acknowledged alert is never aged out | - **Encryption + integrity** for every row above: store cipher + per-value GCM tag, with the cell AAD in - §2 (bound only under `aad_bind = true` / `vault_transit`). + §2 (bound on the shipped `aad_bind = true` default and under `vault_transit`; unbound only where an + operator has set `aad_bind = false`). - **Privacy note.** The two `reason` columns are the one place where a *sensitive* free-text field is readable under a **monitoring-tier** permission rather than a PHI-tier one. That is why they are scrubbed **twice** before the cipher and bounded to 200 characters, and why the tables are documented @@ -852,6 +855,15 @@ connection's `tls_hop_attested`), **synthetic instance → ALLOW**, **clamped gl **enforcing PHI instance → REFUSE (`serve` exits 2)**, **non-enforcing PHI → WARN**. The three named remedies are therefore: native TLS, a loopback agent, or an attested hop. +> **Why this cell still reads the data label, when the transport cells no longer do.** +> [ADR 0153](adr/0153-collapse-the-posture-gradient-no-data-label-may-allow-a-cleartext-hop.md) removed +> the `synthetic → ALLOW` arm from the shared cleartext-hop authority, and its *Explicitly out of scope* +> table keeps this forwarder on the old keying **deliberately**: it is not a connection, so it has +> nowhere to carry a per-hop `cleartext_accepted` declaration, and refusing it instead would create a +> deviation the loosening registry cannot express. The arm is therefore restated explicitly inside +> `forward_hop_disposition` rather than inherited from the authority. A `[logging]` sibling of +> `cleartext_accepted` is the recorded follow-up. + **Availability.** The forwarder never blocks the engine *indefinitely* — UDP is fire-and-forget; a `tcp` **or `tls`** collector that is **unreachable at startup** (or whose certificate fails to verify — `ssl.SSLError` is an `OSError` subclass) is skipped with a warning and the service starts without it, diff --git a/docs/SECURITY-LOOSENING.md b/docs/SECURITY-LOOSENING.md index 0ceef7dc..1d56e95a 100644 --- a/docs/SECURITY-LOOSENING.md +++ b/docs/SECURITY-LOOSENING.md @@ -62,6 +62,26 @@ section reference. | Enforcement dial | `enforcement` | `enforce` (refuse; `warn` = loud audited loosening) | | Posture lever | `handles_real_patient_data` | *derived from environment* | | | `production_instance` | *derived from environment* | +| Outside `[security]` | `[store].aad_bind` | `true` (at-rest values bound to their cell) | +| | `[auth].ad_session_recheck_seconds` | `300` s (*conditional* — a loosening only once `ad_enabled`) | +| Per-connection | `cleartext_accepted` | `false` on every outbound / `FhirLookup` (*connection-scoped* — see below) | + +**Three of these do not live in `[security]`.** `[store].aad_bind` and `[auth].ad_session_recheck_seconds` +sit in their own sections for cohesion, and `cleartext_accepted` is a per-**connection** field, not a +service setting at all. They are listed and reported here anyway, because the rule is *one shipped +posture, loosen only* — a deviation the registry cannot see is a second posture by the back door. The +first two are named by `security_loosenings()` from the loaded `[store]`/`[auth]` sections; the third is +resolved from the loaded connection graph and passed in by name (see its entry below for exactly which +surfaces see it, and which cannot). + +> **Scope, stated plainly.** The registry covers *every* `[security]` switch (a completeness floor in +> `tests/test_security_posture_defaults.py` fails on an unreported, unexempted one) plus the three +> enumerated deviations above. It is **not yet** an exhaustive register of every security-relevant +> switch in every section: `[store].encrypt` / `trust_server_certificate` and +> `[auth].enabled` / `require_mfa` / `ad_tls_verify` / `ad_allow_insecure_ldap` / +> `oidc_require_mfa_claim` are gated by their own serve-time refusals and are **not** reported here. +> That gap is enumerated in the floor test's exemption set, so it is a written decision rather than an +> accident, and a *new* switch in either section cannot join it silently. Closing it is owed work. `enforcement` (ADR 0148 GIVEN 2) is the serve-gate **refuse/warn dial** + the [ADR 0092](adr/0092-posture-keyed-transport-hop-refusal-refuse-the-insecure-phi-hop.md) escape-clamp key, defaulting to `enforce` (byte-identical to the former production-tier refusal). It is @@ -267,6 +287,96 @@ trail. **data-class** declaration, **orthogonal to `enforcement`** — it does not lower the AI data-scope ceiling or re-enable DEBUG-with-PHI logging (both keyed on the retained `production` tier fact, not on `data_class`). +### `[store].aad_bind = false` — at-rest values are no longer bound to their cell +- **What you lose:** the per-value GCM tag stops covering the `(table, column, row)` cell the value lives + in, so a ciphertext **cut and pasted from one cell into another decrypts successfully** instead of + failing its auth tag. An attacker (or a bug) with write access to the store can move a body, a TOTP + secret or an audit detail into a different row and have the engine accept it as that row's content. + Confidentiality is unchanged; what is lost is at-rest **integrity binding** (ASVS 11.3.3). +- **When acceptable:** when you need the frozen `mfenc:v1` at-rest format specifically — a byte-identical + restore target, an external tool that parses the v1 marker, or a forensic comparison against a v1 + backup. It is also a no-op either way with **no `[store].encryption_key`**: the identity cipher has no + tag to bind, so on a keyless store this switch changes nothing. The registry still *reports* it (the + key is env-only and not on `[store]`, so `security_loosenings()` cannot gate on it) — the risk text + carries the caveat instead, so a keyless dev box reads "no effect without a store key" rather than a + weakness it does not have. +- **Compensating controls:** database-level access control (the cell-move attack needs store write + access); `[store].cipher_provider = "vault_transit"`, which binds the AAD **unconditionally** + (`mfenc:v3`) regardless of this switch; the tamper-evident audit chain, which detects reordering of + audit rows independently. +- **Reversible:** yes, in both directions. Legacy `v1` rows always decrypt (dual-read) and + `messagefoundry rotate-key` upgrades them `v1`→`v2` in place, so turning it back on does not strand an + existing store. See [ADR 0019](adr/0019-pluggable-keyprovider-hsm-kms-vault.md) (2026-07-28 amendment). + +### `[auth].ad_session_recheck_seconds = 0` **with `ad_enabled`** — directory revocation stops propagating +> **Conditional**, like `allowed_client_networks`. With no directory to reconcile against, `0` is not a +> weaker choice — it is the only meaningful one — so it is reported as a deviation **only** when +> `[auth].ad_enabled` is true. The shipped `300` default is inert on a non-AD box (the reconciler also +> requires an LDAP client), which is why it does not break one. +- **What you lose:** an AD account that is **disabled or deleted keeps its live engine sessions**. The + only remaining bound is the `[security].max_session_hours` cap (12 h) and idle timeout — so a + terminated employee can hold an authenticated operator session, with PHI access, for up to half a day + after the directory says otherwise. The same loop also revokes on **group-membership change**, so role + removals stop propagating too. +- **When acceptable:** a directory whose service-account bind budget genuinely cannot absorb one bind per + signed-in user per interval; a deployment where operator sessions are already short-lived by policy; or + a break-glass window while a DC problem is diagnosed. Prefer **raising the interval** (it is floored at + 60 s, not capped) over turning it off. +- **Compensating controls:** lower `[security].max_session_hours` and `sign_out_after_idle_minutes` so an + orphaned session expires sooner; revoke sessions manually on offboarding; keep the audit trail + (`auth.ad_session_revoked`) under review. The reconciler is **fail-open** on DC unavailability by + design, so it was never a substitute for these. +- **See:** [ADR 0079](adr/0079-kerberos-idp-session-coordination.md) (2026-07-28 amendment). + +### `cleartext_accepted = true` on a connection — a declared cleartext hop +> **Connection-scoped, unlike every other entry here.** It is not a `[security]` switch; it is a field on +> one connection — an `outbound(...)` or a `FhirLookup(...)` — declared next to the host it governs, with +> a mandatory `cleartext_reason` recorded for the audit trail. +> [ADR 0153](adr/0153-collapse-the-posture-gradient-no-data-label-may-allow-a-cleartext-hop.md). +- **What you lose:** the payload — and any credential that connection carries — crosses that hop + **unencrypted and unauthenticated**, readable and modifiable by anything on the path. There is no + partial protection here: it is plaintext PHI on the wire for that connection. +- **When acceptable:** a peer that genuinely cannot do TLS — vendor firmware that predates it, or a + transport with no TLS support at all. For `Tcp()` and `X12()` the declaration is **permanent and + structural**: those connectors have no `tls` parameter, so there is nothing to migrate to + (BACKLOG #311). For MLLP / HTTP / DICOM / SMTP / FTP it should be **transitional** — it names work to + be done, and it should disappear when the peer gains TLS. +- **Do not use it to describe a hop that *is* secure.** If a proxy terminates TLS in front of the hop, or + the segment is genuinely isolated, that is a different claim entirely — `tls_hop_attested`, which ALLOWs + the hop silently. The two are deliberately separate so the audit trail can tell a proxy-terminated hop + from plaintext on a flat network. Writing an attestation about a hop that is not secure puts a false + statement into the one field that exists to be trustworthy when audited. + **Note (accurate as of 2026-07-28):** `tls_hop_attested` has **no authoring surface on a connection** + today — no transport factory takes it and it is not a `connections.toml` key, so an inbound/outbound + cannot set it (the `[logging].forward_hop_attested` sibling *is* settable). `cleartext_accepted` is + therefore the only per-connection declaration an operator can currently write. Giving attestation an + authoring surface would add a **silent-ALLOW** loosening and needs its own registry entry here first; + it is owed, not shipped. +- **Compensating controls:** network segmentation and physical/link-layer controls on that specific path; + narrow the blast radius by declaring it on the single connection that needs it rather than broadly. +- **It is never silent:** WARN + a dedicated record at **every** connector construction, naming the + declaring connection, the cell, the host and the reason; a `cleartext-accepted` line in + `messagefoundry check` listing the **whole** accepted set (outbound connections *and* `FhirLookup` + read connections); and a `cleartext_accepted` entry in `GET /security/posture`'s loosening list naming + every declaring connection. The construction record is a distinct WARNING **log line**, not a + tamper-evident `audit` table row — the hop decision is pure `config/`-level code and cannot reach the + engine's store across the one-way dependency boundary. The ADR 0092 attestation record has the same + shape for the same reason. +- **Where it is NOT reported, and why:** `messagefoundry security show` reads a settings file and never + loads the connection graph, so it cannot see these declarations; it says so explicitly in its + `loosenings_scope` output rather than reporting a settings-only list as if it were the whole posture. + `GET /security/posture` carries the same `loosenings_scope` marker in the one case it is blind — an + engine with no loaded graph (an embedding, or a query before start); it is `null` on a running engine. + The `serve`-time loosening warning fires before the graph is loaded for the same reason — the + construction gate's own per-connection WARN covers it moments later, at startup, with more detail. +- **What it cannot do:** it never yields ALLOW. An accepted hop is always a WARN, so it can never become + invisible — an accepted risk that stops being visible has stopped being accepted and started being + forgotten. It also cannot relax a hop ADR 0153 does not govern: inbound binds are still decided by the + exposed-gates, and **revocation / weakened-TLS (`verify_tls = false`) refusals are unaffected** — a + verify-off hop is encrypted-but-unauthenticated, not cleartext, so it keeps the clamped + `MEFOR_ALLOW_INSECURE_TLS` escape and this declaration does not reach it. Nor does it reach an SMTP + `AUTH` over cleartext, which is refused outright. + --- ## Standards mapping (ASVS v5.0 · NIST SP 800-53r5 · HIPAA §164.312) @@ -295,6 +405,9 @@ carried from that drive-to-pass, not re-derived here.** | `audit_all_authorization_decisions` | V16 Security Logging and Error Handling | **AU-2** Event Logging · **AU-3** Content of Audit Records | §164.312(b) Audit Controls | | `handles_real_patient_data`, `production_instance` (posture lever) | V13 Configuration (risk-based) | **RA-2** Security Categorization · **AC-6** Least Privilege (risk-based tailoring) | §164.308(a)(1) Risk Analysis / Management | | `enforcement` (refuse/warn dial) | V13 Configuration (secure defaults) | **CM-6** Configuration Settings · **CM-7** Least Functionality (secure-by-default) | §164.308(a)(1) Risk Analysis / Management | +| `[store].aad_bind` (at-rest cell binding) | V11 Cryptography | **SC-28(1)** Cryptographic Protection · **SI-7** Software, Firmware, and Information Integrity | §164.312(c)(1) Integrity · §164.312(a)(2)(iv) Encryption and Decryption | +| `[auth].ad_session_recheck_seconds` (directory revocation propagation) | V7 Session Management · V6 Authentication | **AC-2(3)** Disable Accounts · **AC-12** Session Termination | §164.312(a)(2)(i) Unique User Identification · §164.308(a)(3)(ii)(C) Termination Procedures | +| `cleartext_accepted` (per-connection declared cleartext hop) | V12 Secure Communication | **SC-8** Transmission Confidentiality and Integrity · **SC-8(1)** Cryptographic Protection | §164.312(e)(1) Transmission Security · §164.312(e)(2)(ii) Encryption | > The synthetic-vs-PHI relaxation (a synthetic instance keeps the PHI-only gates relaxed) is **risk-based > tailoring** keyed on `handles_real_patient_data`: an instance carrying no ePHI is out of scope for the diff --git a/docs/SECURITY.md b/docs/SECURITY.md index c5879ce8..c9eb0078 100644 --- a/docs/SECURITY.md +++ b/docs/SECURITY.md @@ -1070,8 +1070,8 @@ one-to-one — that is why the bind/exposure posture occupies two rows and the A | Account state — disabled | `user.disabled` | the account is disabled | **DENY** — no identity is built on **any** plane | n/a | (no knob — an admin action) | | Account state — credential rotation pending | `user.must_change_password` | the flag is set | **CONFINE** — every route but the rotation routes is refused (403 JSON / 303 console / hard WS reject) | n/a | (no knob — set by admin creation and password reset) | | Concurrent session count | the user's live session count at login | count would exceed the cap | **DENY** — this login proceeds; the user's **oldest** session is revoked | 5 sessions, `0` = unlimited | `[auth].max_sessions_per_user` | -| Live directory resolvability — probe strikes | a periodic AD probe of principals that still hold sessions | interval floored at 60 s; **2 consecutive** failed passes (`ad_session_recheck_strikes`); ≤ 200 users (`ad_session_recheck_max_users`) probed per pass, least-recently-probed first. Fail-**open** on DC unavailability (an unreachable DC revokes nothing) | **DENY** by revocation, `auth.ad_session_revoked` audited | **0 = off** (the loop is never created) | `[auth].ad_session_recheck_seconds`, `ad_session_recheck_strikes`, `ad_session_recheck_max_users` | -| Live directory group membership vs. the session's granted roles | the AD groups returned by that same reconciliation probe, mapped through the AD-group→role map | on a **successful (PRESENT)** probe, the mapped role set differs from the account's current roles — a **single** pass, **no** strike accrual (unlike the row above) | **DENY** by revocation of every session for that account (the new roles are persisted first), `auth.ad_session_revoked` with `reason = roles_changed`; charged against the same mass-revoke breaker as an absence | **0 = off** (same loop) | `[auth].ad_session_recheck_seconds` | +| Live directory resolvability — probe strikes | a periodic AD probe of principals that still hold sessions | interval floored at 60 s; **2 consecutive** failed passes (`ad_session_recheck_strikes`); ≤ 200 users (`ad_session_recheck_max_users`) probed per pass, least-recently-probed first. Fail-**open** on DC unavailability (an unreachable DC revokes nothing) | **DENY** by revocation, `auth.ad_session_revoked` audited | **300 s** (the shipped default); `0` disables the loop entirely and is a named loosening | `[auth].ad_session_recheck_seconds`, `ad_session_recheck_strikes`, `ad_session_recheck_max_users` | +| Live directory group membership vs. the session's granted roles | the AD groups returned by that same reconciliation probe, mapped through the AD-group→role map | on a **successful (PRESENT)** probe, the mapped role set differs from the account's current roles — a **single** pass, **no** strike accrual (unlike the row above) | **DENY** by revocation of every session for that account (the new roles are persisted first), `auth.ad_session_revoked` with `reason = roles_changed`; charged against the same mass-revoke breaker as an absence | **300 s** (same loop; `0` disables it) | `[auth].ad_session_recheck_seconds` | | Live directory mass-revoke breaker | the size of one pass's revocation set vs the probed population | the set exceeds **both** `ad_session_revoke_max` (**5**) **and** `ad_session_revoke_max_fraction` (**0.34**) — a second **binary** predicate layered on the row above, never a score (see "Directory session reconciliation") | **LOG** — the pass aborts revoking **nothing**, logs at ERROR and writes an `auth.ad_reconcile_aborted` audit row + loud alert | 5 / 0.34 | `[auth].ad_session_revoke_max`, `ad_session_revoke_max_fraction` | | PHI-read volume, per actor | `identity.user_id` | > 120 reads (`phi_read_rate_limit_per_actor`) per 60 s (`phi_read_rate_limit_window_seconds`); the global dimension `phi_read_rate_limit_global` defaults to `0` = **off** | **THROTTLE** 429 + `Retry-After: 10`, WARNING-logged, charged at **admission** before any store work | on, 120 / 60 s | `[auth].phi_read_rate_limit_enabled` | | Admin-write rate, per actor | `identity.user_id` × request method | **non-GET only**; > 12 writes (`admin_write_rate_limit_per_actor`) per 1.0 s (`admin_write_rate_limit_window_seconds`); no global dimension (`glob=0`) | **THROTTLE** 429 + `Retry-After: 1`, WARNING-logged. **JSON API only** — no `/ui` route charges it at this release | on, 12 writes / 1.0 s | `[auth].admin_write_rate_limit_enabled` | @@ -1294,7 +1294,7 @@ was everything with no step-up gate: **bulk and raw PHI reads** (`GET /messages` attachments, `/dead-letters` — paced at `[auth].phi_read_rate_limit_per_actor`, 120/min) and **connection start/stop/restart**. -`[auth].ad_session_recheck_seconds` (**default `0` = off**) starts a background pass that re-resolves +`[auth].ad_session_recheck_seconds` (**default `300` s**; `0` = off) runs a background pass that re-resolves every directory principal still holding a live session — via the same password-free service-account lookup the Kerberos path uses — and revokes the sessions of accounts AD has disabled or deleted. Group membership is re-diffed on the same pass at no extra directory cost, so a **role demotion** takes effect @@ -1318,7 +1318,7 @@ Three safety properties, because the lookup returns one indistinguishable "not f Revocation is therefore bounded by *interval × strikes* (10 minutes at the recommended 300 s), not immediate, and one LDAP bind per signed-in directory user per pass is the cost — `ad_session_recheck_max_users` (200) caps it, and it is zero when nobody is signed in. An off-loopback -PHI deployment serving AD accounts should set `ad_session_recheck_seconds = 300`. +PHI deployment serving AD accounts gets `ad_session_recheck_seconds = 300` by default; setting it to `0` is a declared loosening, not a neutral choice. ### Session inventory & targeted revocation (WP-10) @@ -1396,7 +1396,7 @@ Comparative properties on the dimensions the table's four columns cannot carry: | Pathway | Phishing resistance | Replay resistance | Credential stored by the engine | MFA support | Revocation | |---|---|---|---|---|---| | **Local** | passkeys only (WebAuthn origin-bound, `attestation=none`, `user_verification=preferred`); password/TOTP are phishable | TOTP is single-use per 30 s step (`totp_skew_steps` default `0`); recovery codes single-use; passkey challenges are 64-byte CSPRNG, single-use, 120 s TTL, with a strict sign-counter compare-and-set | argon2id password hash (t=3, m=64 MiB, p=4); TOTP secret **cipher-encrypted**; recovery codes argon2id-hashed; COSE public keys **plaintext by design** | built (TOTP + passkeys) | disable the account or revoke sessions — immediate | -| **AD** | none | none beyond TLS | **none** — only the service-account bind password (env or a `[secrets]` reference, fail-closed) | delegated and **unverifiable** — the simple-bind leg issues MFA-satisfied under a signed relaxation; the engine receives no evidence (contrast OIDC's signature-verified `amr`/`acr`, which is enforced) | disabling in AD does **not** end a live session; `[auth].ad_session_recheck_seconds` (default **0 = off**) closes it, bounded by interval × strikes | +| **AD** | none | none beyond TLS | **none** — only the service-account bind password (env or a `[secrets]` reference, fail-closed) | delegated and **unverifiable** — the simple-bind leg issues MFA-satisfied under a signed relaxation; the engine receives no evidence (contrast OIDC's signature-verified `amr`/`acr`, which is enforced) | disabling in AD does **not** end a live session on its own; `[auth].ad_session_recheck_seconds` (default **300 s**) closes it, bounded by interval × strikes | | **Kerberos** | none (single-leg, no channel binding) | ticket lifetime is the domain's | **none** — the acceptor keytab/SPN is OS-owned | delegated, unverifiable | as AD | | **OIDC** | the IdP's, not the engine's | strongest of the four: server-side PKCE verifier + `state` (constant-time compare) + `nonce`, single-use flow, a `__Host-`-prefixed browser-binding cookie the callback requires, and a kid/alg/signature/`iss`/`aud`/`exp`/`iat`/`nbf` ladder under a bounded clock skew | **none** — only the confidential-client secret (env-only or a `[secrets]` reference, resolved eagerly at startup) | asserted via `amr`/`acr` **and enforced** — with `[auth].oidc_require_mfa_claim` on (default) a token carrying no configured `amr`/`acr` is refused at claims validation, and only then is the session minted MFA-verified; switch it off and the federated session is minted **un**verified, which `mfa_satisfied` refuses. This is the one directory leg whose factor the engine actually verifies | as AD, plus the `id_token.exp` cap; no refresh tokens and no RP-initiated logout | | **mTLS** | n/a (no interactive ceremony) | n/a | **none** — the engine holds only the pinned client CA and the name map | none, structurally | **no revocation checking** — `VERIFY_X509_STRICT` is strict path validation, not OCSP/CRL; live revocation is the org's PKI. Engine-side: remove the allow-list entry (config change → restart) or disable the mapped account | diff --git a/docs/adr/0019-pluggable-keyprovider-hsm-kms-vault.md b/docs/adr/0019-pluggable-keyprovider-hsm-kms-vault.md index 3a4a0aa3..91d41129 100644 --- a/docs/adr/0019-pluggable-keyprovider-hsm-kms-vault.md +++ b/docs/adr/0019-pluggable-keyprovider-hsm-kms-vault.md @@ -481,6 +481,9 @@ reuse the same `hvac` extra. ## Amendment 2026-07-17 — cell-bound AES-GCM AAD at rest is now BUILT, opt-in (ASVS 11.3.3, ADR 0115 / WP #244) +> **Superseded in part (2026-07-28):** the `[store].aad_bind` default described below as **off** is +> now **on**. See *Amendment 2026-07-28* at the end of this ADR. Everything else in this section stands. + **BUILT (opt-in, default off) — supersedes the deferred-hardening posture below.** ASVS 11.3.3 (bind associated data into the at-rest AEAD tag) is now a shipped, tested control on the `mfenc:v2` writer, gated by a new `[store].aad_bind` knob (env `MEFOR_STORE_AAD_BIND`; default **off**, so the at-rest @@ -666,3 +669,30 @@ and the in-process bulk-crypto DEK-in-heap deferred to 11.7.1 / WP-BL3-28). The [ASVS-L3-ASSESSMENT.md](../security/ASVS-L3-ASSESSMENT.md) / [ASVS-FAILS-REMEDIATION-PLAN.md](../security/ASVS-FAILS-REMEDIATION-PLAN.md) / [PHI.md](../PHI.md) §4/§11 / [CONFIGURATION.md](../CONFIGURATION.md) / [SECURITY.md](../SECURITY.md) row updates are the **Coordinator's** single-writer task — this ADR edits no score doc.* + +## Amendment 2026-07-28 — `[store].aad_bind` now defaults ON (ADR 0148 GIVEN 1) + +**What changed.** `[store].aad_bind` flips `false` → **`true`**. New at-rest AES-256-GCM writes use the +cell-bound `mfenc:v2` writer by default; the frozen `mfenc:v1` writer is now the *opt-out*. + +**Why.** ADR 0148 GIVEN 1: there is exactly ONE shipped posture, and it is the hardened one. A control +that ships off is a control first exercised in production, by the deployment least able to absorb a +surprise. Keeping the hardened path as the default means it is the path every test, every CI leg and +every dogfood instance runs. + +**Why it is safe to flip.** The format change is additive and dual-read: legacy `mfenc:v1` rows still +decrypt unchanged, `messagefoundry rotate-key` upgrades them `v1`→`v2` in place, and `aad_bind` is a +no-op without an encryption key (the identity cipher has nothing to bind). An existing store therefore +keeps working across the upgrade, and the flip is reversible by setting the knob back. + +**What it costs.** CRYPTO-1's "byte-identical at rest" property no longer describes the *default* — it +describes `aad_bind = false`. That is the deliberate trade: byte-identity was a migration-safety +property, and dual-read already provides the safety it was protecting. + +**Visibility.** `aad_bind = false` is a **loosening**, not a neutral choice: `security_loosenings()` +names it, so it appears in the serve-time warning and in `GET /security/posture`, and it has an entry in +[docs/SECURITY-LOOSENING.md](../SECURITY-LOOSENING.md). Setting it is allowed; setting it silently is +not. + +**Scope.** This amends the *default* only. The `cell_aad` builder, the `mfenc:v2` format, the dual-read +dispatch, the rotation upgrade path and the three-backend threading are all unchanged. diff --git a/docs/adr/0079-kerberos-idp-session-coordination.md b/docs/adr/0079-kerberos-idp-session-coordination.md index b575a32f..62f4fcb9 100644 --- a/docs/adr/0079-kerberos-idp-session-coordination.md +++ b/docs/adr/0079-kerberos-idp-session-coordination.md @@ -260,3 +260,33 @@ cannot reach everyone rotates least-recently-probed-first, degrading to a longer - **Not changed.** `_mfa_required_for` still returns False for every non-LOCAL provider — AD MFA stays delegated to the directory (ADR 0002). Local sessions, the loopback default, and every path with `ad_session_recheck_seconds = 0` are byte-identical. + +## Amendment 2026-07-28 — `[auth].ad_session_recheck_seconds` now defaults to 300 (ADR 0148 GIVEN 1) + +**What changed.** `[auth].ad_session_recheck_seconds` flips `0` (off) → **`300`** (five minutes), the +value this ADR and `docs/SECURITY.md` already recommended for an off-loopback PHI deployment. Everything +else about mechanism 2 — the 60 s floor, `ad_session_recheck_strikes`, `ad_session_recheck_max_users`, +the mass-revoke breaker, the fail-open-on-DC-unavailability posture — is unchanged. + +**Why.** ADR 0148 GIVEN 1: the hardened path is the shipped path. Left at `0`, directory revocation did +not propagate at all — an AD account disabled or deleted kept its live engine sessions until the +`[security].max_session_hours` cap. A recommendation that must be typed to take effect is a control that +is off in most deployments. + +**Why the shipped default does not break a non-AD deployment.** The loop requires an LDAP client: +`AuthService.should_reconcile()` checks for one, so a deployment that never enables `ad_enabled` creates +no task and issues no bind. The default is therefore **inert** without AD. + +**The cross-field refusal is re-keyed, not removed.** `ad_session_recheck_seconds` without `ad_enabled` +used to fail startup — correctly, because an operator who typed it believed directory revocation now +propagated, and a silently-dead security control is worse than one never enabled. With a non-zero +*shipped* default that rule would fail startup on every non-AD box. It is now keyed on +`model_fields_set`: an **explicitly configured** value without `ad_enabled` still refuses (the case the +rule exists for), while an untouched default — which carries no operator belief to falsify, and is inert +anyway — does not. + +**Visibility.** `ad_session_recheck_seconds = 0` **with** `ad_enabled` is a **loosening**: +`security_loosenings()` names it, so it appears in the serve-time warning and in +`GET /security/posture`, with an entry in [docs/SECURITY-LOOSENING.md](../SECURITY-LOOSENING.md). It is +deliberately conditional — with no directory to reconcile against, `0` is not a weaker choice, it is the +only meaningful one, so it is not reported as a deviation on a non-AD instance. diff --git a/docs/adr/0092-posture-keyed-transport-hop-refusal-refuse-the-insecure-phi-hop.md b/docs/adr/0092-posture-keyed-transport-hop-refusal-refuse-the-insecure-phi-hop.md index 0833ecfb..06e90a7b 100644 --- a/docs/adr/0092-posture-keyed-transport-hop-refusal-refuse-the-insecure-phi-hop.md +++ b/docs/adr/0092-posture-keyed-transport-hop-refusal-refuse-the-insecure-phi-hop.md @@ -1,5 +1,23 @@ # ADR 0092 — Posture-keyed transport-hop refusal (refuse the insecure PHI hop) +> **AMENDED by [ADR 0153](0153-collapse-the-posture-gradient-no-data-label-may-allow-a-cleartext-hop.md) +> (built 2026-07-28) — decisions 1 and 2.** The precedence below is no longer current: arm 3 +> (`not is_phi → ALLOW`) is **deleted**, so `insecure_hop_disposition` no longer takes `is_phi`, and the +> `audited_opt_out` arm went with it, so `MEFOR_ALLOW_INSECURE_TLS` can no longer influence a +> cleartext-hop decision. The live precedence is: loopback → ALLOW, `hop_attested` → ALLOW, +> `cleartext_accepted` → WARN, not `enforcing` → WARN, else REFUSE. +> +> Everything else in this ADR **stands**: the one-authority structure, the loopback carve-out +> (decision 1 arm 1), the attestation (decision 3), the two-layer construction/send gating (decision 4) +> and the no-loosen rule (decision 5) — which 0153 preserves *by construction*, since the only deleted +> arm returned ALLOW. `MEFOR_ALLOW_INSECURE_TLS` itself is unhooked from the **cleartext** decision, not +> deleted. It survives for the non-connection cells (engine→store TLS, LDAPS, the webhook alert sink, the +> AI broker, the `[logging]` forwarder and the API PHI-read serve hop) that have nowhere to carry a +> per-connection declaration, **and for the weakened-TLS cells** — `verify_tls=false` on +> REST/SOAP/FHIR/DICOMweb, `tls_verify=false` on MLLP and FTPS, `trust_server_certificate` on the store. +> A verify-off hop is encrypted-but-unauthenticated rather than cleartext, so it sits outside 0153's +> stated scope: it keeps this ADR's clamped escape unchanged, and `cleartext_accepted` does not reach it. + **Status:** Accepted (2026-07-11) — owner-ratified design (a prior design pass + an adversarial security critic set the decisions below; they are not re-litigated here). CORE built (BACKLOG #200): the shared authority + escape clamp + per-hop attestation field + posture threading. The transport **cells** that diff --git a/docs/adr/0153-collapse-the-posture-gradient-no-data-label-may-allow-a-cleartext-hop.md b/docs/adr/0153-collapse-the-posture-gradient-no-data-label-may-allow-a-cleartext-hop.md index ca26def2..68ff29d7 100644 --- a/docs/adr/0153-collapse-the-posture-gradient-no-data-label-may-allow-a-cleartext-hop.md +++ b/docs/adr/0153-collapse-the-posture-gradient-no-data-label-may-allow-a-cleartext-hop.md @@ -1,8 +1,9 @@ # ADR 0153 — Collapse the posture gradient: no data label may allow a cleartext hop **Status:** Accepted (2026-07-25) -- owner-ratified after an adversarial review returned REWORK on the first -draft; the three open questions the rework raised were answered the same day and are folded in below. NOT yet -built. Amends +draft; the three open questions the rework raised were answered the same day and are folded in below. +**BUILT 2026-07-28** (see *Build notes* at the end for the three implementation questions this ADR left +open and how they were resolved). Amends [ADR 0092](0092-posture-keyed-transport-hop-refusal-refuse-the-insecure-phi-hop.md) decisions 1 and 2 — specifically the `is_phi` ALLOW arm and the escape clamp. Keeps 0092's one-authority structure, its loopback carve-out, its attestation (decision 3), its two-layer construction/send gating (decision 4) and its @@ -150,7 +151,7 @@ These read the authority or its posture and are **unchanged** by this ADR -- a r |---|---| | `phi_read_hop_disposition` (`tls_policy.py:501-531`) | the API PHI-read serve hop is not a connection; it hardcodes `hop_attested=False` and has no per-connection field to carry a declaration. Stays keyed on posture. | | `settings.forward_hop_disposition` (`settings.py:2304-2317`) | the `[logging]` syslog/SIEM forwarder defaults to plaintext UDP (`settings.py:1315`) and is not a connection. Stays keyed on posture; a `[logging]` sibling of `cleartext_accepted` is a follow-up. | -| `rest._shipped_strict_disposition` (`rest.py:296-318`) | governs shipped-strict REST behaviour, not a cleartext hop decision. | +| `rest._shipped_strict_disposition` (`rest.py:296-318`) | **AMENDED AT BUILD (2026-07-28) — see Build note 1.** Its *floor* is not reworked, but the floor's KEY moved from the global escape to `cleartext_accepted` for the cleartext cells it serves, because a literal port would have made decision 2 inert for the whole HTTP family. The `verify_tls=false` cell it also serves is genuinely unchanged: it keeps the clamped global escape and does **not** take `cleartext_accepted`. | | `revocation_hop_disposition` (`tls_policy.py:551-590`) | ADR 0078 / #201 revocation gate; reads `HopPosture`'s fields, which this ADR retains. | | `_inbound_insecure_bind_permitted` (`wiring_runner.py:6100-6124`) | the inbound bind gate; keyed on `--allow-insecure-bind` + `tls_hop_attested`, untouched (see decision 2). | | `weakened_tls_escape_permitted` | reads posture fields, which are retained. | @@ -172,8 +173,13 @@ These read the authority or its posture and are **unchanged** by this ADR -- a r ### Negative / risks * **A synthetic-data instance loses its blanket carve-out.** A test instance sending to a cleartext sink must - now declare `cleartext_accepted` per destination, or run with `[security].enforcement = warn`. This is more - ceremony than before, and it is the cost of the change. + now declare `cleartext_accepted` per destination. This is more ceremony than before, and it is the cost of + the change. **Correction (build, 2026-07-28):** the alternative this bullet also offered — "or run with + `[security].enforcement = warn`" — works only for the raw transports. The HTTP family + (REST/SOAP/FHIR/DICOMweb/`FhirLookup`) shipped these refusals unconditionally, so + `rest._shipped_strict_disposition`'s ADR 0092 §5 no-loosen floor turns that WARN back into a REFUSE + there. The per-connection declaration is the only route for those cells, which is exactly why Build + note 1 re-keys the floor on it. * **Raw TCP and X12 can never satisfy the gate.** Their declaration is permanent (decision 4). An operator reading only decision 2 might expect a migration path that does not exist for them. * **`cleartext_accepted` can be applied broadly.** Nothing stops an operator declaring it on every @@ -211,3 +217,110 @@ hop is secure" about a hop that is not, and the attestation's only value is bein **Delete `MEFOR_ALLOW_INSECURE_TLS` outright.** Cleanest end state, rejected for now — six non-connection cells have no other expressible escape, and it would break roughly 17 CI legs and the container manifests in a change that is supposed to be about the hop authority. + +## Build notes (2026-07-28) + +Built as specified. Five questions the ADR did not settle had to be answered to implement it; each is +recorded here because leaving them to an implementer's silent choice is exactly how a scope decision +becomes an accident. (Notes 1 and 3 were corrected on 2026-07-28 after an adversarial review — the +first draft of note 1 over-reached, and note 3 claimed a threading that had not been done. Notes 4 and +5 record what that review found.) + +**1. `rest._shipped_strict_disposition`'s no-loosen floor is re-keyed on `cleartext_accepted` — for its +CLEARTEXT cells only.** The floor read `if disposition is WARN and not audited_opt_out: return REFUSE`, +which is how `MEFOR_ALLOW_INSECURE_TLS` relaxed an HTTP-family cleartext hop. A naive port would have +left `if disposition is WARN: return REFUSE`, converting decision 2's WARN straight back to REFUSE and +making `cleartext_accepted` **inert** for REST, SOAP, FHIR, DICOMweb, the HTTP credential cells and the +`fhir_lookup` read path — the largest cleartext-egress family in the product, and the only one where the +declaration is a genuine escape (`Tcp()`/`X12()` never reach that cell). For those cells the floor is now +keyed on `cleartext_accepted`, which preserves 0092 §5 exactly (a hop reaching WARN via the non-enforcing +dial alone is still floored to REFUSE, as today) and makes decision 2 effective where it matters. The +stated side effect: an instance that set `MEFOR_ALLOW_INSECURE_TLS` to cross a non-enforcing HTTP +**cleartext** hop no longer can. That is a **tightening**, and it is what decision 5 asks for. + +The same function also serves the **`verify_tls=false`** cell, and that one is NOT re-keyed. A verify-off +hop is encrypted-but-unauthenticated, not cleartext, so this ADR — scoped to "the cleartext transport-hop +decision only" — does not govern it. It keeps the pre-0153 clamped global escape (`weakened_tls=True` in +`_shipped_strict_disposition`), exactly as the MLLP and FTPS `tls_verify=false` cells do through +`weakened_tls_escape_permitted_here()`. Threading `cleartext_accepted` into it was tried and **reverted**: +it would have LOOSENED a hop that refuses on an enforcing instance today (0092 decision 5 forbids that), +attached an operator's written "this peer cannot do TLS" reason to a peer that plainly does TLS, and split +the HTTP family from MLLP on the same question. The only 0153 change that reaches the verify-off cell is +the deleted `not is_phi` ALLOW arm, which can only tighten it. Pinned by +`tests/test_rest_transport.py::test_rest_verify_tls_false_not_relaxed_by_a_cleartext_declaration`. + +**2. The two out-of-scope *delegating* callers restate the deleted arm explicitly.** +`phi_read_hop_disposition` and `settings.forward_hop_disposition` are the only out-of-scope consumers +that **call** the authority rather than reading `HopPosture` directly, so dropping `is_phi` would have +silently taken their synthetic-ALLOW arm with it — turning a synthetic instance's plaintext-UDP +`[logging]` forwarder and its non-loopback API PHI-read hop into refusals under `enforce`, contradicting +this ADR's own "keeps its inputs and its behaviour". Each now carries `if not posture.is_phi: return +ALLOW` before delegating, and passes the clamped global escape as the new arm-3 argument (byte-identical +— arm 3 occupies exactly the slot the old arm 4 did). Both are non-connections with nowhere to carry a +declaration, so refusing them would create a deviation the loosening registry cannot express. Restated, +not inherited: the scope limit is now a written decision at the one place it applies. + +**3. `cleartext_accepted` reaches the CREDENTIAL hops as well as the body hops.** The ADR is silent, and +putting a password on the wire is a materially worse claim than putting a body on it. It is threaded +anyway — to HTTP Digest, the OAuth2 **and SMART** token endpoints, the forward-proxy credential and the +SOAP WS-Security / body-secret cells — because the alternative leaves an operator whose legacy peer needs +Basic auth over a cleartext segment with no honest declaration, and therefore pushes them toward writing +a **false `tls_hop_attested`**: precisely the defect this ADR exists to remove. SMTP AUTH over cleartext +remains refused **outright** in `transports/email.py`; that is a hard refusal, not a posture decision, +and is untouched. + +The **SMART** token endpoint (`transports/smart.py`) needed more than threading. It did not consume the +hop authority at all: it read the raw, *unclamped* `insecure_tls_allowed()`, so one process-wide +environment variable put a signed `client_assertion` on cleartext http even on an enforcing PHI instance, +and that escape appeared in no loosening registry. It is now routed through +`refuse_cleartext_credential_hop`, exactly like its OAuth2 sibling — a **tightening**, and the last place +`MEFOR_ALLOW_INSECURE_TLS` was still alive on a connection-scoped cleartext decision. + +**4. `FhirLookup()` gained the declaration as a real parameter, and the loosening reader walks the +lookups.** A `FhirLookup` connection has no `Destination`, and its read executor honours the pair off the +spec settings. Left there alone, the only way to declare it would have been mutating `spec.settings` by +hand: an escape with no load validation and no entry in `accepted_cleartext_hops`, so a live cleartext +**PHI-read** hop could cross while `check`, `security_loosenings()` and `GET /security/posture` all +reported the accepted set as empty. The pair is now a `FhirLookup()` parameter, coherence-checked at that +one authoring surface, and `accepted_cleartext_hops` walks `registry.fhir_lookups` as well as +`registry.outbound` (lookup entries are named `fhir_lookup:`, a separate namespace). + +**5. Found while building, NOT fixed here: `tls_hop_attested` has no authoring surface on a +connection.** No transport factory takes it, it is not a `connections.toml` key, and it is absent from +`_INBOUND_KEYS` / `_OUTBOUND_KEYS` — the connectors read it off `Destination`/`Source`, but the only way +to populate it is constructing the model by hand. So the ALLOW arm this ADR keeps (decision 1 arm 2) is +unreachable from config today, and decision 2's framing — "using attestation for a legacy peer writes a +false statement" — describes a mistake an operator currently *cannot* make on a connection. (The +`[logging].forward_hop_attested` sibling in `messagefoundry.toml` IS settable, and the retro-fitted +mandatory-reason rule bites there.) + +This was left unbuilt deliberately. Wiring it would ADD a **silent-ALLOW** per-connection loosening, and +under "one shipped posture, loosen only" that needs its own `docs/SECURITY-LOOSENING.md` entry and its +own owner ratification — it is not a correction to this ADR's build. What was corrected is the +documentation: `docs/CONNECTIONS.md` had begun listing `tls_hop_attested = true` as one of "exactly +three ways such a hop crosses", so an operator following it got a hard load error and the only row that +loaded was `cleartext_accepted` — pushing them to declare "this hop is NOT secure" about a hop that is, +inverting the one distinction this ADR says the audit trail exists to preserve. Both operator docs now +say plainly that attestation is present in the engine and unreachable from config. **Owed.** + +**Also built, beyond the decision list.** The retro-fitted flag-implies-reason rule on +`tls_hop_attested` (decision 2) reaches `[logging].forward_hop_attested` too — it shares +`_check_hop_attestation`, is documented as "the `[logging]` sibling", and `docs/PHI.md` already described +its reason as mandatory, so scoping the rule away from it would have left a documented guarantee +unenforced exactly where an auditor would look. `mllp.InsecureHopGuard`'s and +`rest._enforce_shipped_hop`'s attestation-audit branches dropped their `posture.is_phi` conjunct: with +the authority no longer reading the label, gating the audit on it would have silenced the record for the +very hops that newly depend on an attestation to cross. + +**Visibility surfaces (owner requirement).** A declared acceptance appears in: a WARN plus a dedicated +record at **every** connector construction, naming the declaring connection, the cell, the host and the +reason; the `cleartext-accepted` line of `messagefoundry check`, which lists the whole accepted set; and +the `cleartext_accepted` entry in `security_loosenings()` / `GET /security/posture`, naming every +declaring connection. The construction record is a distinct WARNING log line, not a `store.record_audit` +row — the decision helper is pure `config/`-level code and cannot reach the engine's store across the +one-way dependency boundary (the ADR 0092 attestation record has the same shape and the same reason). + +Three surfaces cannot see the whole set, and each SAYS so rather than reporting a subset: +`messagefoundry security show` and `GET /security/posture` on an engine with no loaded graph both emit a +`loosenings_scope` marker; the `serve`-time loosening warning fires before the graph is loaded and says +the construction gate reports them separately (it does, moments later, per connection). diff --git a/docs/adr/README.md b/docs/adr/README.md index f150f878..f685c3d2 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -174,4 +174,4 @@ recommended in [Secure Development Standards §5](../Secure_Development_Standard | [0150](0150-client-address-on-audit-entries.md) | **Client address on audit entries** — an audit row named *who* and *what* but never *where from*, so after an incident there was no way to tell which host performed an action (e.g. pulled a bulk PHI export). The two apparent answers both fail: the one address-bearing event (WP-L3-13 `auth.admin_action_new_ip`) is emitted **only** from the four step-up dependencies — the raw-PHI view uses `require_phi_read` and never reaches it — and is default-off (`[auth].admin_new_ip_step_up`) + change-only; and `sessions.client` holds the address captured at **login**, so on a **replayed token** it names the original victim's host — *actively misleading*, not merely lossy. Adds a nullable `client` column to `audit_log` on **all three backends** (SQLite/PG `TEXT`, SQL Server `NVARCHAR(256)`, each backend's own additive-migration idiom). **The address is folded into the CHAINED payload** (an unchained sibling column an attacker could rewrite without breaking tamper-evidence would be worse than none) **as a CONDITIONAL trailing element** — appended only when non-`None`, so a clientless row (every legacy row, every engine-internal write) hashes over the same 6-element list and its `row_hash` still verifies **byte-identically**; an *unconditional* 7th element would have appended `null` to every legacy payload and broken the chain at the first row on upgrade. Composes with the #190 keyed-chain watermark (keying picks SHA-256 vs HMAC; this picks the bytes). Threaded **explicitly** (`client=`) — a `ContextVar` was **rejected**: it leaks across `asyncio.create_task` and would stamp a live operator's address onto unrelated `system` rows. `NULL` = *no client in scope*, never inherited (summary coalescer, dual-control executor, console callback keep NULL by design). One extraction path (`_client_ip` → public `client_ip`); teed off-box as a discrete SIEM field; surfaced on `GET /audit` + the CSV export. Plaintext at rest by decision (integrity-protected via the chain; matches `sessions.client`) — recorded in the [PHI.md](../PHI.md) at-rest inventory | Accepted (2026-07-22) — built; pushes/PR owner-approved | | [0151](0151-operator-surface-source-network-allow-list-security-allowed-client-networks.md) | **Operator-surface source-network allow-list (`[security].allowed_client_networks`)** — a CIDR allow-list for the operator API + web console, enforced in the OUTERMOST ASGI middleware (before routing, dependencies, the body cap and auth; covers `/ui`, the `/ui/static` mount and the `/ws/stats` WebSocket). **Empty = the default = no restriction, byte-identical to today.** Turns "the console is reachable only from the hospital network" from a firewall-only assumption into a config assertion the engine enforces, `GET /security/posture` reports and a test verifies. **D-1:** it evaluates `scope["client"]` and parses NO forwarding header — uvicorn's `ProxyHeadersMiddleware` is the single XFF trust point and already rewrote it from `[api].trusted_proxies`, so a declared proxy (R2) yields the real client and an undeclared one (R1) means an attacker's XFF is ignored outright; **R3** (proxy in front, nothing declared) leaves the control **INERT** — an honest limit, pinned by a test, detected by an address-monoculture tripwire and never to be documented as covered. **D-2:** loopback is allowed UNCONDITIONALLY (the XFF-conditional variant would lock the on-box operator out of the recommended proxy-on-the-engine-box topology); the ingest `source_ip_allowlist` does NOT inherit the carve-out. **D-3:** setting the allow-list REFUSES a multi-address `[api].trusted_proxies` entry — any host in a trusted range can forge its own XFF, so a `10.0.0.0/8` entry would silently nullify the restriction just asked for. **D-4:** the loosening advisory keys on EXPOSURE, not the bind, so it fires in R2. **D-5:** diagnosability is part of the control (marker header, address-echoing 403 page, rate-limited log, posture counters, `/health` left exempt and echoing `observed_client`). Matcher hoisted to a neutral stdlib-only `messagefoundry/netaddr.py` shared with the ingest allow-list. Defence-in-depth BEHIND the host firewall, never the primary network control; does not cover ingest, a confused deputy on an allowed subnet, or NAT; startup-only (a lockout costs a restart). IDE field deferred; #26-clean | Accepted (2026-07-22) — built + green; default-off | | [0152](0152-in-use-data-protection-for-phi-platform-memory-encryption-attestation-asvs-11-7-1.md) | **In-use data protection for PHI — platform memory-encryption attestation (ASVS 11.7.1)** — the only Fail on the 2026-07-22 single-posture scorecard. 11.7.1 is **L3-only and net-new in ASVS 5.0** (`ADDED`, no 4.0.3 lineage), so L1+L2 is untouched (zero Fails); we pursue it because L3 is the target for a PHI system. **Decision: the engine MEASURES and REPORTS, the deployment DETERMINES the verdict** — no Pass claimed from documentation, configuration, or a local capability flag. Three rungs: (1) platform read-out surfaced in `GET /security/posture` beside the FIPS attestation (ADR 0120), report-only and explicitly NOT satisfying the requirement; (2) `[security].memory_encryption_operator_declared`, on the `MEFOR_TLS_REVOCATION_ATTESTED` pattern — the rung that moves **Fail → Partial**. **Amended 2026-07-22:** a missing declaration on an exposed PHI instance **warns**; the refusal is opt-in via `[security].require_memory_encryption_declaration` (default off), because ADR 0148 makes even `dev` derive PHI, "exposed" includes the recommended loopback-behind-proxy topology, and no operator can satisfy the property on Windows — a default refusal would have hard-stopped deployments that boot today, which the ADR 0151 opt-in-scoping rule forbids. Named `operator_declared`, NOT `attested`: in confidential computing "attestation" means the rung-3 signed quote, and `"memory_encryption_attested": true` is exactly the quotable false-compliance artifact this ADR exists to prevent. The posture body carries its own disclaimer (`memory_encryption_note`), and the read-out is never accepted as a substitute for the declaration; (3) CPU-signed quote verified against vendor root PKI (SEV-SNP `/dev/sev-guest`, TDX `/dev/tdx_guest`) with an operator-supplied CACHED vendor chain because air-gapped is the norm here — the only rung supporting **Partial → Pass**, and deployment-gated on SEV-SNP/TDX hardware. Memory HYGIENE is explicitly not this requirement: OWASP deleted 4.0.3's V8.3.6 memory-zeroing as NOT PRACTICAL and kept 11.7.1, and our shipped mlock/zeroize (#198) moves the cell by zero. No cloud-provider SDKs (on-prem product); The Windows spike RAN (2026-07-22) and its answer is recorded in the ADR: Windows rung 3 is **platform-blocked, not API-blocked** — a vTPM NV-index path exists under a Microsoft paravisor (Azure CVMs) and needs no new dependency, but no on-prem Hyper-V/ESXi boots a Windows confidential guest at all, so **on-prem Windows is capped at rungs 1+2 = Partial**, disclosed. It also corrects the Linux framing: `/dev/sev-guest` is absent on Azure Linux CVMs too, so rung 3 is two acquisition backends (direct guest device | vTPM NV index) × two OSes, not "Linux vs Windows". Residual that survives all rungs: plaintext PHI and the unwrapped DEK remain in CPython heap, protected from host/hypervisor but not from in-guest code execution. **Built: rungs 1 + 2 (report-only read-out + the operator declaration) and the deployment requirements; rung 3 is NOT built, so no Pass is reachable today on any platform.** Accepting the ADR deliberately does NOT re-score 11.7.1 — that is an owner decision on the assessment of record. Plan: docs/security/ASVS-11-7-1-IN-USE-DATA-PLAN.md | Accepted (2026-07-22) | -| [0153](0153-collapse-the-posture-gradient-no-data-label-may-allow-a-cleartext-hop.md) | **Collapse the posture gradient — no data label may allow a cleartext hop** — ADR 0092's precedence arm 3 is `not is_phi -> ALLOW`, so an instance whose environment file declares `data_class = "synthetic"` crosses **every** cleartext transport hop silently, with no warning and no audit record. Found by upgrading a dogfood instance whose environment file had been quietly unparseable: the moment it loaded and `data_class = "phi"` took effect, a cleartext MLLP egress that had been crossing without comment became a refusal — nothing about the hop had changed, only a label in a different file, and nothing had reported the guard was off. Deletes that one arm: `insecure_hop_disposition` loses `is_phi` and the precedence becomes loopback -> ALLOW, attested -> ALLOW, `cleartext_accepted` -> WARN, not-`enforcing` -> WARN, else REFUSE. Because the deleted arm returned ALLOW, removing it can only turn a crossing into a WARN or a REFUSE, so **0092's no-loosen rule holds by construction** and an unchanged config is byte-identical unless it relied on the label. Adds `cleartext_accepted` + `cleartext_reason` on **Destination**, deliberately SEPARATE from `tls_hop_attested`: attestation claims *this hop IS secure by means the engine cannot see* and passes silently, the new pair admits *it is NOT, and we accept that* and warns + audits every construction — merging them would leave the audit trail unable to tell a proxy-terminated hop from plaintext PHI on a flat network. **No TLS default is flipped** (an earlier draft proposed it): redundant, since an undeclared cleartext outbound already REFUSES and a refusal naming the hop beats a handshake failure; it hard-fails 14 of 14 inbound MLLP listeners because `_mllp_ssl_context(server=True)` demands a cert before any policy runs, unreachable by posture/attestation/the new flag and not loopback-exempt; and it would silently disarm the four inbound exposed-gates, which early-return when `tls` is truthy. `Tcp()`/`X12()` have no TLS support at all, so for them the declaration is **permanent and structural**, not transitional. Neither `data_class` nor `MEFOR_ALLOW_INSECURE_TLS` is removed — the hop authority stops READING the label, and the variable is unhooked from the authority but survives for the six non-connection cells (store TLS, LDAPS, webhook sink, AI broker) that have no per-connection field to carry an escape. Retains ADR 0148's `[security].enforcement` dial as an arm (its AC-3 requires it on every gate). Scope is the cleartext transport-hop decision ONLY; an explicit out-of-scope table names all four callers of the authority and the three other readers of `HopPosture` | Accepted (2026-07-25) — reworked after adversarial review; owner-ratified; NOT yet built | +| [0153](0153-collapse-the-posture-gradient-no-data-label-may-allow-a-cleartext-hop.md) | **Collapse the posture gradient — no data label may allow a cleartext hop** — ADR 0092's precedence arm 3 is `not is_phi -> ALLOW`, so an instance whose environment file declares `data_class = "synthetic"` crosses **every** cleartext transport hop silently, with no warning and no audit record. Found by upgrading a dogfood instance whose environment file had been quietly unparseable: the moment it loaded and `data_class = "phi"` took effect, a cleartext MLLP egress that had been crossing without comment became a refusal — nothing about the hop had changed, only a label in a different file, and nothing had reported the guard was off. Deletes that one arm: `insecure_hop_disposition` loses `is_phi` and the precedence becomes loopback -> ALLOW, attested -> ALLOW, `cleartext_accepted` -> WARN, not-`enforcing` -> WARN, else REFUSE. Because the deleted arm returned ALLOW, removing it can only turn a crossing into a WARN or a REFUSE, so **0092's no-loosen rule holds by construction** and an unchanged config is byte-identical unless it relied on the label. Adds `cleartext_accepted` + `cleartext_reason` on **Destination**, deliberately SEPARATE from `tls_hop_attested`: attestation claims *this hop IS secure by means the engine cannot see* and passes silently, the new pair admits *it is NOT, and we accept that* and warns + audits every construction — merging them would leave the audit trail unable to tell a proxy-terminated hop from plaintext PHI on a flat network. **No TLS default is flipped** (an earlier draft proposed it): redundant, since an undeclared cleartext outbound already REFUSES and a refusal naming the hop beats a handshake failure; it hard-fails 14 of 14 inbound MLLP listeners because `_mllp_ssl_context(server=True)` demands a cert before any policy runs, unreachable by posture/attestation/the new flag and not loopback-exempt; and it would silently disarm the four inbound exposed-gates, which early-return when `tls` is truthy. `Tcp()`/`X12()` have no TLS support at all, so for them the declaration is **permanent and structural**, not transitional. Neither `data_class` nor `MEFOR_ALLOW_INSECURE_TLS` is removed — the hop authority stops READING the label, and the variable is unhooked from the authority but survives for the six non-connection cells (store TLS, LDAPS, webhook sink, AI broker) that have no per-connection field to carry an escape. Retains ADR 0148's `[security].enforcement` dial as an arm (its AC-3 requires it on every gate). Scope is the cleartext transport-hop decision ONLY; an explicit out-of-scope table names all four callers of the authority and the three other readers of `HopPosture` | Accepted (2026-07-25) — reworked after adversarial review; owner-ratified; **BUILT 2026-07-28** | diff --git a/messagefoundry/__main__.py b/messagefoundry/__main__.py index e48cff5e..355b140e 100644 --- a/messagefoundry/__main__.py +++ b/messagefoundry/__main__.py @@ -1432,12 +1432,19 @@ def _serve(args: argparse.Namespace) -> int: # plain language, so a deliberate opt-out is never silent. Advisory only — the posture GATES below # still refuse a production-PHI weakening (the ADR 0092 clamp is unchanged). The shared # security_loosenings() feeds both this warning and the read-only GET /security/posture view. - _loosenings = security_loosenings(settings.security) + # The connection graph is NOT loaded yet here (the Engine loads it inside the ASGI lifespan, well + # below), so this early warning covers the SETTINGS-scoped switches only and passes an empty + # cleartext-hop list. That is not a silent subset: every ADR 0153 acceptance is reported moments + # later — per connection, with its reason — by the construction gate's own loud WARN + audit record, + # and completely by `messagefoundry check` and GET /security/posture, which both have the graph. + _loosenings = security_loosenings(settings.security, settings.store, settings.auth, ()) if _loosenings: _seclog = logging.getLogger(__name__) _seclog.warning( "[security] posture loosened from the secure defaults (%d): %s — see " - "docs/SECURITY-LOOSENING.md. Production-PHI weakenings are still refused below.", + "docs/SECURITY-LOOSENING.md. Production-PHI weakenings are still refused below. " + "Per-connection cleartext_accepted declarations (ADR 0153) are reported separately by " + "the connector construction gate.", len(_loosenings), "; ".join(f"{name} ({risk})" for name, risk in _loosenings), ) @@ -4170,12 +4177,54 @@ def _security(args: argparse.Namespace) -> int: from pydantic import ValidationError from messagefoundry.config import security_edit - from messagefoundry.config.settings import SecuritySettings, load_settings, security_loosenings + from messagefoundry.config.settings import ( + AuthSettings, + SecuritySettings, + StoreSettings, + load_settings, + security_loosenings, + ) path = args.service_config + # This subcommand edits [security], but security_loosenings() also reports [store]/[auth] deviations + # (ADR 0148: one posture). Resolve those from the whole file so the list is complete. If the file will + # not load — it may be invalid OUTSIDE [security], which must not break `security show` — fall back to + # the shipped defaults and SAY SO via the emitted `loosenings_partial` marker, rather than silently + # reporting a subset as if it were everything. + _loosenings_partial = False + _store, _auth = StoreSettings(), AuthSettings() + if Path(path).exists(): + # An ABSENT file is not a degraded read — the shipped defaults ARE the effective posture there, + # and `security show` is expected to work offline before any config exists. Only a file that + # exists and will not resolve is partial. + try: + _full = load_settings(config_path=path) + _store, _auth = _full.store, _full.auth + except (ValidationError, tomllib.TOMLDecodeError, OSError, ValueError): + # The specific ways a settings file fails to resolve: a schema/cross-field violation, + # malformed TOML, an unreadable path, and the plain ValueErrors load_settings raises for a + # bad env/section. Anything else is a programming error and must surface, not be degraded + # into a boolean. + _loosenings_partial = True + def _loosenings(sec: SecuritySettings) -> list[dict[str, str]]: - return [{"switch": s, "risk": r} for s, r in security_loosenings(sec)] + # This CLI reads a SETTINGS file and never loads the connection graph, so it cannot see the ADR + # 0153 per-connection cleartext_accepted declarations — it passes an empty list and declares the + # gap in `loosenings_scope` below, instead of reporting a settings-only view as if it were the + # whole posture. `messagefoundry check` and GET /security/posture are the complete surfaces. + return [{"switch": s, "risk": r} for s, r in security_loosenings(sec, _store, _auth, ())] + + #: Emitted alongside every loosening list this subcommand prints, so a reader can never mistake a + #: degraded or settings-only report for a complete one. `partial` means [store]/[auth] could not be + #: read at all (the file did not load); `connections_not_loaded` is the standing limitation above. + _loosenings_scope = { + "loosenings_partial": _loosenings_partial, + "loosenings_scope": ( + "settings only ([security]/[store]/[auth]); per-connection cleartext_accepted " + "declarations are NOT included — see `messagefoundry check` or GET /security/posture" + ), + } if args.action == "show": try: @@ -4191,6 +4240,7 @@ def _loosenings(sec: SecuritySettings) -> list[dict[str, str]]: "set": sorted(raw.keys()), "defaults": SecuritySettings().model_dump(), "loosenings": _loosenings(resolved), + **_loosenings_scope, }, compact=args.json, ) @@ -4221,6 +4271,7 @@ def validate(settings_path: Path) -> None: return _emit_error(f"invalid [security] value: {exc}", as_json=args.json) result = security_edit.set_security(path, updates, validate=validate) result["loosenings"] = _loosenings(SecuritySettings.model_validate(merged)) + result.update(_loosenings_scope) except json.JSONDecodeError as exc: return _emit_error(f"invalid security update JSON: {exc}", as_json=args.json) except (security_edit.SecurityEditError, FileNotFoundError, ValueError, OSError) as exc: diff --git a/messagefoundry/api/_ui_seam.py b/messagefoundry/api/_ui_seam.py index 7d96ee6c..1eeb488f 100644 --- a/messagefoundry/api/_ui_seam.py +++ b/messagefoundry/api/_ui_seam.py @@ -69,7 +69,13 @@ #: fields satisfies 11.7.1, and the status page renders them worded "self-reported", never "compliant". #: (The last three were renamed/retyped from an earlier draft of THIS seam — v13 is unreleased, so the #: field set is corrected in place rather than burning v14 on a shape no console ever saw.) -ENGINE_UI_SEAM: int = 14 +#: seam v15: SecurityPosture gained the additive `loosenings_scope` — `None` when the loosening list is +#: COMPLETE, else a string naming what it could not see (an engine with no loaded connection graph +#: cannot read the ADR 0153 per-connection `cleartext_accepted` declarations). Additive with a default, +#: so an older console simply ignores it; bumped rather than corrected in place because v14 SHIPPED +#: (v0.3.2). Under "one shipped posture, loosen only" a subset that reads as the whole posture is the +#: failure this field exists to prevent, so the console must be able to render the caveat. +ENGINE_UI_SEAM: int = 15 @dataclass(frozen=True, slots=True) diff --git a/messagefoundry/api/app.py b/messagefoundry/api/app.py index 11dd0ca0..64c72ca4 100644 --- a/messagefoundry/api/app.py +++ b/messagefoundry/api/app.py @@ -246,6 +246,7 @@ EnvRef, Registry, WiringError, + accepted_cleartext_hops, load_config, redacted_settings, ) @@ -1467,7 +1468,13 @@ async def security_posture( cipher via the public ``store.cipher_info()`` accessor (never the private ``_cipher``), and ``key_source`` is the provider *name*. ``plaintext_columns`` reports any PHI column left unencrypted on the active backend — empty on every backend now (the SQL Server residual was - retired by H4; SQLite/Postgres/SQL Server all have full at-rest coverage).""" + retired by H4; SQLite/Postgres/SQL Server all have full at-rest coverage). + + **Engine-shard scope (ADR 0037).** The connection-scoped part of ``loosenings`` (the ADR 0153 + ``cleartext_accepted`` declarations) is read off THIS process's registry, which in a sharded + deployment is the shard-filtered graph — so each shard reports its own declared set, not the + estate's. ``messagefoundry check`` reads the whole config dir and is the estate-wide surface. + ``loosenings_scope`` is non-``None`` when this engine has no loaded graph at all.""" # The live cipher posture (on/off + key fingerprint only). cipher_info() is the public Store # accessor — the route never touches engine.store._cipher. info = engine.store.cipher_info() @@ -1481,9 +1488,32 @@ async def security_posture( # notice. security is the resolved SecuritySettings the serve path stashed (defaults on the # test/embedding path). No secret material — these are booleans/ints only. security = getattr(request.app.state, "security", None) or SecuritySettings() + # [store]/[auth] carry posture switches too (ADR 0148: one posture, loosen only), so the registry + # needs them to report a COMPLETE list. Same stash-or-default pattern as `store` above. + auth_settings = getattr(request.app.state, "auth_settings", None) or AuthSettings() + # ADR 0153: the ONE connection-scoped deviation. Read LIVE off the running graph (so a reload is + # reflected) — this route is where an operator learns a cleartext hop is being crossed by + # declaration, and a stale or absent list would understate the posture. An engine with no + # registry runner (an embedding, or an app queried before start) cannot see them at all, so it + # DECLARES that in `loosenings_scope` rather than returning a settings-only subset that reads as + # the whole posture — the same discipline `messagefoundry security show` follows. + runner = engine.registry_runner + cleartext_hops = ( + [name for name, _ in accepted_cleartext_hops(runner.registry)] + if runner is not None + else [] + ) + loosenings_scope = ( + None + if runner is not None + else ( + "settings only — no connection graph is loaded on this engine, so per-connection " + "cleartext_accepted declarations are NOT included (see `messagefoundry check`)" + ) + ) loosenings = [ SecurityLoosening(switch=name, risk=risk) - for name, risk in security_loosenings(security) + for name, risk in security_loosenings(security, store, auth_settings, cleartext_hops) ] synthetic_relaxation = ( "strict PHI-only controls (at-rest-encryption refusal, deny-by-default egress, bounded " @@ -1510,7 +1540,10 @@ async def security_posture( "backend": backend, "encryption_enabled": info.encrypts, "key_source": store.key_provider, - "loosenings": [name for name, _ in security_loosenings(security)], + # Reuse the list computed above rather than recomputing it: a second call could + # observe a different graph after a concurrent reload, and the audit row must record + # exactly what the response reports. + "loosenings": [entry.switch for entry in loosenings], } ), client=client_ip(request), @@ -1529,6 +1562,7 @@ async def security_posture( plaintext_columns=_plaintext_columns(backend, encryption_enabled=info.encrypts), security=security.model_dump(), loosenings=loosenings, + loosenings_scope=loosenings_scope, synthetic_relaxation=synthetic_relaxation, fips_mode=fips_mode, # interpreter ssl/_hashlib OpenSSL FIPS-provider state; None=undeterminable openssl_version=openssl_version, # that OpenSSL's version string (public metadata) @@ -5483,6 +5517,13 @@ async def _audit_upload_prune(meta: UploadedFileMeta) -> None: reconciler: asyncio.Task[None] | None = None bootstrap_reminder: asyncio.Task[None] | None = None security_notifier = None + # Back the COMPLETE loosening list on GET /security/posture: [auth] carries posture switches + # (ad_session_recheck_seconds) that security_loosenings() must see. Stashed here, OUTSIDE the + # `enabled` guard below, deliberately — a settings object that exists but is disabled is still + # the resolved settings, and stashing it only on the enabled path would make the route silently + # fall back to AuthSettings() defaults and report a subset. Mirrors store_settings above. + if auth_settings is not None: + app.state.auth_settings = auth_settings if auth_settings is not None and auth_settings.enabled: # Out-of-band security-event push (#188, ASVS 6.3.5/6.3.7) — reuses the [alerts] SMTP # transport, sent to each affected user's own address. The notifier is wired only when the diff --git a/messagefoundry/api/models.py b/messagefoundry/api/models.py index 78e01cc2..6b74265b 100644 --- a/messagefoundry/api/models.py +++ b/messagefoundry/api/models.py @@ -924,6 +924,12 @@ class SecurityPosture(BaseModel): # PHI-only gates are (defensibly) relaxed. Read-only; the IDE is the sole authoring surface. security: dict[str, object] = Field(default_factory=dict) loosenings: list[SecurityLoosening] = Field(default_factory=list) + # ``None`` = the loosening list above is COMPLETE. A string names what it could NOT see — set only + # when this engine has no loaded connection graph (an embedding, or an app queried before start), + # where the ADR 0153 per-connection ``cleartext_accepted`` declarations are unreadable. Reporting a + # settings-only subset with no marker would understate the posture, which is the one thing this + # route must not do; ``messagefoundry security show`` carries the same marker for the same reason. + loosenings_scope: str | None = None # Set WHERE handles_real_patient_data=false: the strict PHI-only controls (at-rest-encryption refusal, # deny-by-default egress, bounded retention) are relaxed because the instance carries no ePHI (AC-6). synthetic_relaxation: str | None = None diff --git a/messagefoundry/checks.py b/messagefoundry/checks.py index a13cd2da..73d14ba4 100644 --- a/messagefoundry/checks.py +++ b/messagefoundry/checks.py @@ -152,6 +152,9 @@ def run_checks( service_config=service_config, suppress_search=suppress_service_toml_search, ), + # ADR 0153: name every outbound that declares cleartext_accepted, so the accepted set is visible + # in review rather than discoverable only by reading each connection. Advisory — see the check. + _check_cleartext_accepted(config_dir), ] if run_lint: results.append(_run_tool("ruff", ["ruff", "check", str(config_dir)])) @@ -1311,6 +1314,52 @@ def _check_build( ) +def _check_cleartext_accepted( + config_dir: str | Path, +) -> CheckResult: + """Surface **the whole set** of connections that declare ``cleartext_accepted`` (ADR 0153). + + ADR 0153 accepts, and cannot prevent, an operator declaring the acceptance broadly enough to + approximate the blanket escape it removed. Its stated mitigations are that the declaration is + per-connection, that it warns and audits at every construction, and that **``check`` surfaces the + whole accepted set** — this is that surface. Advisory (``required=False``): a declared acceptance is + a legitimate, reasoned choice, not a config error, and blocking on it would push operators back + toward a false ``tls_hop_attested``. It exists so the set is *visible in review*, next to the hosts. + + Covers outbound connections AND ``FhirLookup`` read connections — ``accepted_cleartext_hops`` walks + both, so the "whole accepted set" claim is true rather than nearly true. + + SKIPs when the graph will not load — ``validate`` reports that, and a check that silently reported + an empty accepted set on an unloadable config would be worse than one that says it could not look.""" + from messagefoundry.config.wiring import WiringError, accepted_cleartext_hops, load_config + + try: + registry = load_config(config_dir) + except (WiringError, OSError, ImportError, SyntaxError, ValueError) as exc: + return CheckResult( + "cleartext-accepted", + ok=True, + required=False, + skipped=True, + detail=f"config did not load: {exc}", + ) + accepted = accepted_cleartext_hops(registry) + if not accepted: + return CheckResult( + "cleartext-accepted", + ok=True, + required=False, + detail="no connection declares cleartext_accepted", + ) + listed = "; ".join(f"{name} ({reason})" for name, reason in accepted) + return CheckResult( + "cleartext-accepted", + ok=True, + required=False, + detail=(f"{len(accepted)} connection(s) cross a cleartext hop by declaration — {listed}"), + ) + + def _check_reference_backend( config_dir: str | Path, *, diff --git a/messagefoundry/config/connections_edit.py b/messagefoundry/config/connections_edit.py index 86e263fc..b978f4c9 100644 --- a/messagefoundry/config/connections_edit.py +++ b/messagefoundry/config/connections_edit.py @@ -86,6 +86,12 @@ # DR priority tier (#61, ADR 0048) + engine-shard partition tag (ADR 0037/0073). "priority", "shard", + # ADR 0153: the per-outbound cleartext-hop acceptance (outbound-only). It MUST be written back — + # silently dropping it on the next GUI/CLI save would turn a declared, reviewed, audited cleartext + # hop into a REFUSED one at the next reload, and the operator would watch a lane fail for a reason + # nothing in their edit explains. `cleartext_reason` rides with it; the pair is validated together. + "cleartext_accepted", + "cleartext_reason", ) # Sub-tables follow the scalars. ``schedule`` is deliberately LAST — it nests ``windows`` as an array # of inline tables, visually the heaviest entry, so the simple knobs stay greppable above it. diff --git a/messagefoundry/config/connections_file.py b/messagefoundry/config/connections_file.py index 780a8644..f93fb1a3 100644 --- a/messagefoundry/config/connections_file.py +++ b/messagefoundry/config/connections_file.py @@ -140,6 +140,12 @@ # Cosmetic "Waiting for Reply" pre-display delay (#136, ADR 0065 amendment) — default 0.0, so an # existing table without it is byte-identical. Display-only; no delivery effect. "waiting_display_delay", + # ADR 0153 decision 2: the per-outbound cleartext-hop acceptance. TOP-LEVEL keys (as the ADR's + # TOML sample shows them), NOT under [settings] — a hop *policy* declaration belongs beside the + # connection's other governance keys, and [settings] is the transport factory's own schema + # (see _build_spec: "the factory IS the schema"), which no factory would accept. + "cleartext_accepted", + "cleartext_reason", } ) @@ -251,6 +257,11 @@ def _outbound_from_table(table: dict[str, Any], source: str) -> OutboundConnecti flagged=_require_bool(table, "flagged", where, default=False), # #136: the cosmetic waiting-for-reply pre-display delay; absent = 0.0 (show immediately). waiting_display_delay=_optional_float(table, "waiting_display_delay", where) or 0.0, + # ADR 0153: the cleartext-hop acceptance pair; absent = off, so an existing table is + # byte-identical. The flag/reason coherence rules live once in build_outbound_connection, so + # this surface and the code-first outbound() surface cannot drift. + cleartext_accepted=_require_bool(table, "cleartext_accepted", where, default=False), + cleartext_reason=_optional_str(table, "cleartext_reason", where), source_file=source, source_line=None, ) diff --git a/messagefoundry/config/models.py b/messagefoundry/config/models.py index 3be2bf13..e4d91259 100644 --- a/messagefoundry/config/models.py +++ b/messagefoundry/config/models.py @@ -175,21 +175,63 @@ class InternalErrorPolicy(str, Enum): # noqa: UP042 def _check_hop_attestation(attested: bool, reason: str | None) -> None: - """Load-validate the per-connection insecure-hop attestation pair (#200, ADR 0092). - - A ``tls_hop_attested_reason`` is only meaningful alongside ``tls_hop_attested=true`` — a reason - without the flag is a config mistake (the operator meant to attest but didn't), so it fails loud at - load. A blank/whitespace-only reason is likewise rejected: an attestation that suppresses a would-be - production-PHI refusal should carry a real justification for the audit trail.""" + """Load-validate the per-connection insecure-hop attestation pair (#200, ADR 0092 / 0153). + + Three rules, all fail-loud at load: + + #. a reason without the flag is a config mistake (the operator meant to attest but didn't); + #. a blank/whitespace-only reason is rejected; + #. **the flag without a reason is rejected** (ADR 0153 decision 2, retro-fitted). An attestation + asserting *this hop is secure by means the engine cannot see* is precisely the claim that most + needs a written justification when it is audited — it is the one input that can silently ALLOW + an enforcing cleartext hop. Nothing is deployed, so there is no config to migrate. + + Shared verbatim with ``[logging].forward_hop_attested`` (``settings.LoggingSettings``), which + re-raises under its own field names. Rule 3 reaches that sibling **deliberately**: it is documented + as "the ``[logging]`` sibling of a connection's ``tls_hop_attested``" and ``docs/PHI.md`` already + described its reason as mandatory, so scoping the rule away from it would keep a documented + guarantee un-enforced in exactly the place an auditor would look.""" if reason is not None and not attested: raise ValueError( "tls_hop_attested_reason is set without tls_hop_attested=true — set the flag to attest " "the hop is secure, or drop the reason" ) + if attested and reason is None: + raise ValueError( + "tls_hop_attested=true requires tls_hop_attested_reason — an attestation that this hop is " + "secure by means the engine cannot see must record WHY, for the audit trail (ADR 0153)" + ) if attested and reason is not None and not reason.strip(): raise ValueError("tls_hop_attested_reason must be non-empty when provided") +def _check_cleartext_acceptance(accepted: bool, reason: str | None) -> None: + """Load-validate the per-outbound cleartext-acceptance pair (ADR 0153 decision 2). + + The exact inverse of :func:`_check_hop_attestation`, with the opposite claim: ``tls_hop_attested`` + says *this hop IS secure by means the engine cannot see* (ALLOW, silent); ``cleartext_accepted`` + says *this hop is NOT secure and we accept that* (WARN, logged + audited at every construction). + They are deliberately separate fields — collapsing them would leave the audit trail unable to + distinguish a proxy-terminated hop from plaintext PHI on a flat network, which is the one + distinction it exists to preserve. + + Same three fail-loud rules: the flag without a reason, a blank/whitespace-only reason, and a reason + without the flag. The engine can check a reason is *present and non-blank*; it cannot check that it + is *true* — a placeholder reason is a review problem, not a load problem.""" + if reason is not None and not accepted: + raise ValueError( + "cleartext_reason is set without cleartext_accepted=true — set the flag to accept the " + "cleartext hop, or drop the reason" + ) + if accepted and reason is None: + raise ValueError( + "cleartext_accepted=true requires cleartext_reason — an accepted risk that stops being " + "visible has stopped being accepted and started being forgotten (ADR 0153)" + ) + if accepted and reason is not None and not reason.strip(): + raise ValueError("cleartext_reason must be non-empty when provided") + + class Source(BaseModel): """An inbound connector endpoint.""" @@ -560,6 +602,18 @@ class Destination(BaseModel): # even on production-PHI. Default False → keyed purely on posture (existing outbounds byte-identical). tls_hop_attested: bool = False tls_hop_attested_reason: str | None = None + # ADR 0153 decision 2 — the HONEST escape for a peer that cannot do TLS, and the exact opposite + # claim to tls_hop_attested above: "this hop is NOT secure, and we accept that". It yields WARN, + # never ALLOW — the hop is crossed, but loudly logged at EVERY construction and audited, because an + # accepted risk that stops being visible has stopped being accepted. DESTINATION-ONLY: inbound + # binds are governed by _inbound_insecure_bind_permitted + the four exposed-gates, so the same pair + # on Source would be a setting nothing consumes. For Tcp()/X12() — which have no TLS support at all + # — this is a PERMANENT, STRUCTURAL declaration, not a transitional one: there is no `tls = true` + # for them to migrate to (BACKLOG #311). Default off → nothing changes shape until an operator + # declares something. Surfaced as a loosening (security_loosenings / GET /security/posture) and + # listed by `messagefoundry check`, so the accepted set is never invisible. + cleartext_accepted: bool = False + cleartext_reason: str | None = None # #201 (ADR 0078 amendment): per-connection attestation that a revocation-checking PKI backs a # VERIFYING outbound TLS hop (MLLP-over-TLS / https REST-SOAP-FHIR). The engine performs no OCSP/CRL # (stdlib ssl has none), so an off-loopback production-PHI verified hop is REFUSED at construction / @@ -585,6 +639,7 @@ class Destination(BaseModel): @model_validator(mode="after") def _validate_hop_attestation(self) -> Destination: _check_hop_attestation(self.tls_hop_attested, self.tls_hop_attested_reason) + _check_cleartext_acceptance(self.cleartext_accepted, self.cleartext_reason) return self diff --git a/messagefoundry/config/settings.py b/messagefoundry/config/settings.py index 12a7012b..731627fa 100644 --- a/messagefoundry/config/settings.py +++ b/messagefoundry/config/settings.py @@ -25,7 +25,7 @@ import re import string import tomllib -from collections.abc import Mapping +from collections.abc import Mapping, Sequence from datetime import date from enum import Enum from pathlib import Path @@ -194,12 +194,18 @@ def hop_insecure_escape_downgrades(*, enforcing: bool) -> bool: The **clamp** on the blunt global escape for the posture-keyed hop refusal (ADR 0092, decision 2): the escape may only relax a hop REFUSE to WARN when the security dial is **not enforcing**. Under - ENFORCE it is **inert** for hop refusal — it can NEVER satisfy an enforcing PHI hop (a deliberate - behaviour change from the pre-#200 global escape, which silenced the refusal in every environment). - Cells pass the result as - :func:`~messagefoundry.config.tls_policy.insecure_hop_disposition`'s ``audited_opt_out`` argument, - so under ENFORCE that argument is always ``False`` and the ``enforcing`` REFUSE arm wins. - Attestation (per-connection ``tls_hop_attested``) is the only way to cross an enforcing PHI hop.""" + ENFORCE it is **inert** — it can NEVER satisfy an enforcing hop (a deliberate behaviour change from + the pre-#200 global escape, which silenced the refusal in every environment). + + **Scope after ADR 0153 (decision 5): the TRANSPORT cells no longer consult this at all.** The + cleartext-hop authority lost its ``audited_opt_out`` parameter, so the variable cannot influence a + connection's cleartext-hop decision; the per-connection ``cleartext_accepted`` declaration replaced + it there. This clamp survives for the two **non-connection** cells that still key on the escape and + have nowhere to carry a per-hop declaration — the ``[logging]`` forwarder + (:func:`forward_hop_disposition`) and the API PHI-read serve hop + (:func:`~messagefoundry.config.tls_policy.phi_read_hop_disposition`) — plus the verify-off cells via + :func:`weakened_tls_escape_permitted`. On a connection, ``tls_hop_attested`` (ALLOW) or + ``cleartext_accepted`` (WARN + audit) is now the only way across an enforcing hop.""" return insecure_tls_allowed() and not enforcing @@ -364,13 +370,15 @@ class StoreSettings(_Section): # the env key takes precedence. Empty = use `encryption_key` (the cross-platform default). encryption_key_file: str | None = None # Bind each at-rest AES-256-GCM value to its (table, column, row) cell via GCM Associated Data - # (ASVS 11.3.3, ADR 0019). Off by default → the frozen mfenc:v1 writer, byte-identical at rest - # (CRYPTO-1). When true, NEW writes use the mfenc:v2 writer with cell-bound AAD (it sets the cipher's - # `write_v2`), so a ciphertext cut-and-pasted into another cell fails the auth tag (dead-lettered, - # not silently accepted); legacy v1 rows still decrypt (dual-read) and `messagefoundry rotate-key` - # upgrades them v1→v2. No effect without an encryption key (the identity cipher has nothing to bind). - # The hardened (Posture B) setting — see docs/security/OFF-LOOPBACK-DEPLOYMENT.md. - aad_bind: bool = False + # (ASVS 11.3.3, ADR 0019). **On by default** (ADR 0148 GIVEN 1: the default configuration runs the + # hardened path, so it is exercised everywhere and not first in production): NEW writes use the + # mfenc:v2 writer with cell-bound AAD (it sets the cipher's `write_v2`), so a ciphertext cut-and-pasted + # into another cell fails the auth tag (dead-lettered, not silently accepted). Legacy v1 rows still + # decrypt (dual-read) and `messagefoundry rotate-key` upgrades them v1→v2, so the flip is safe on an + # existing store and reversible. No effect without an encryption key (the identity cipher has nothing + # to bind). Setting it false selects the frozen mfenc:v1 writer (byte-identical at rest, CRYPTO-1) and + # is a LOOSENING — `security_loosenings()` names it, so the opt-out is never silent. + aad_bind: bool = True # KeyProvider seam (ADR 0019, ASVS 13.3.3): selects HOW the active/retired DEK bytes are *sourced* — # never how they are used (the cipher, keyring, and `mfenc:v1` format are unchanged). `auto` (the # default) is the env-then-DPAPI ladder, BYTE-IDENTICAL to the pre-seam behavior; `env`/`dpapi` pin a @@ -1770,11 +1778,17 @@ class AuthSettings(_Section): # principal that still holds a live session and revokes the sessions of accounts that have been # disabled or deleted. See docs/adr/0079-kerberos-idp-session-coordination.md. # - # How often a reconciliation pass runs, in seconds. **0 = OFF, the default** — the loop is never - # created and an upgrade is byte-identical. A non-zero value is floored at 60 s: the pass costs one - # LDAP bind per signed-in directory user, and a fat-fingered `1` would be a DC denial-of-service. - # 300 (five minutes) is the recommended setting for an off-loopback PHI deployment. - ad_session_recheck_seconds: int = 0 + # How often a reconciliation pass runs, in seconds. **300 (five minutes) by default** (ADR 0148 + # GIVEN 1 — the hardened path is the shipped path), floored at 60 s: the pass costs one LDAP bind per + # signed-in directory user, and a fat-fingered `1` would be a DC denial-of-service. `0` disables the + # loop entirely and is a LOOSENING once AD is enabled — `security_loosenings()` names it. + # + # The default is INERT without AD: `AuthService.should_reconcile()` also requires an LDAP client, so a + # deployment that never enables `ad_enabled` creates no task and issues no bind. That is why the + # cross-field check below refuses only an EXPLICIT non-zero value without `ad_enabled` — refusing the + # shipped default would break every non-AD deployment at startup, while an operator who deliberately + # typed a value still gets told their control would be dead. + ad_session_recheck_seconds: int = 300 # How many CONSECUTIVE passes must fail to find a principal before its sessions are revoked. A # single ambiguous result never revokes: `resolve_principal` collapses "disabled", "deleted" and # "the search returned nothing" into one `None`, so requiring two agreeing probes costs at most one @@ -2034,9 +2048,19 @@ def _require_ad_fields(self) -> AuthSettings: ) if self.kerberos_enabled and not self.ad_enabled: raise ValueError("kerberos_enabled requires ad_enabled (SSO resolves roles via AD)") - if self.ad_session_recheck_seconds and not self.ad_enabled: + if ( + self.ad_session_recheck_seconds + and not self.ad_enabled + and "ad_session_recheck_seconds" in self.model_fields_set + ): # Refuse rather than no-op: an operator who set this believes directory revocation now # propagates. A silently-dead security control is worse than never having enabled it. + # + # Keyed on model_fields_set, not on the value, since the hardened SHIPPED default (300, ADR + # 0148 GIVEN 1) is non-zero: refusing it unconditionally would fail startup on every + # deployment that does not use AD, which is most of them. An untouched default carries no + # operator belief to falsify, and it is inert anyway — should_reconcile() also requires an + # LDAP client. An explicitly typed value still refuses, which is the case the rule is for. raise ValueError( "ad_session_recheck_seconds requires ad_enabled (the reconciler re-resolves " "principals through the same LDAP service-account bind)" @@ -2297,37 +2321,50 @@ def forward_hop_disposition(log: LoggingSettings, posture: HopPosture) -> HopDis CA-anchored), so a secure transport is available and this is a *default* problem, not a capability gap. - The decision is delegated verbatim to :func:`~messagefoundry.config.tls_policy. - insecure_hop_disposition` — the SAME authority the transports consume — so the forwarder decides - identically to every other egress cell. A hop is treated as **secure** (and never gated) only when - it is TLS *with verification on*; plaintext ``udp``/``tcp`` and the ``forward_tls_verify=false`` - opt-out are both MITM-able and go to the gradient: + The decision is delegated to :func:`~messagefoundry.config.tls_policy.insecure_hop_disposition` — + the SAME authority the transports consume — so the forwarder decides identically to every other + egress cell. A hop is treated as **secure** (and never gated) only when it is TLS *with + verification on*; plaintext ``udp``/``tcp`` and the ``forward_tls_verify=false`` opt-out are both + MITM-able and go to the gradient: #. loopback collector → ALLOW — the ADR 0080 "point ``tcp``/``udp`` at ``127.0.0.1`` and let a local rsyslog/Vector agent add TLS" deployment is explicitly preserved, byte-identical. + #. synthetic instance (not ``is_phi``) → ALLOW — silent, nothing sensitive rides the hop. Applied + HERE (not by the shared authority, which ADR 0153 stripped of the label) — see below. #. ``forward_hop_attested`` → ALLOW — the acknowledged, reasoned opt-out (a trusted management segment), the ``[logging]`` sibling of a connection's ``tls_hop_attested``. - #. synthetic instance (not ``is_phi``) → ALLOW — silent, nothing sensitive rides the hop. #. the CLAMPED global escape → WARN (never fires under ENFORCE — see :func:`hop_insecure_escape_downgrades`). #. enforcing PHI → REFUSE. #. else (non-enforcing PHI) → WARN. Callers that have not resolved a posture pass the fail-closed one; ``serve`` supplies - :func:`hop_posture_from_ai`. Pure so the gate is unit-testable without standing up ``serve``.""" + :func:`hop_posture_from_ai`. Pure so the gate is unit-testable without standing up ``serve``. + + **ADR 0153 leaves this cell keyed on the data label, deliberately** (its *Explicitly out of scope* + table: "Stays keyed on posture; a ``[logging]`` sibling of ``cleartext_accepted`` is a follow-up"). + The forwarder is not a connection, so it has nowhere to carry a per-hop declaration, and refusing + it instead would create a deviation the loosening registry cannot express. The ``not is_phi`` ALLOW + arm 0153 deleted from the shared authority is therefore restated HERE, explicitly, rather than + inherited — the scope limit is a written decision at the one place it applies, not an emergent + property of a signature change.""" if log.forward_protocol is SyslogProtocol.TLS and log.forward_tls_verify: # Verified, CA-anchored TLS (ADR 0080) — an encrypted+authenticated hop, nothing to gate. return HopDisposition.ALLOW + if not posture.is_phi: + # ADR 0153 scope carve-out — see the docstring. Restated here, not inherited. + return HopDisposition.ALLOW return insecure_hop_disposition( - is_phi=posture.is_phi, enforcing=posture.enforcing, # An unset forward_host cannot happen with forwarding on (the validator requires it), and the # empty string is treated as loopback by the shared predicate — so an unconfigured forwarder # can never be refused. is_loopback_hop=is_loopback_hop_host(log.forward_host or ""), hop_attested=log.forward_hop_attested, - # Clamped upstream to non-enforcing, so under ENFORCE this is always False and the blunt - # global escape can never cross an enforcing PHI hop (ADR 0092 decision 2). - audited_opt_out=hop_insecure_escape_downgrades(enforcing=posture.enforcing), + # The global escape keeps its arm HERE (same scope carve-out): it is the only expressible + # relaxation this non-connection cell has. Clamped upstream to non-enforcing, so under ENFORCE + # it is always False and can never cross an enforcing PHI hop (ADR 0092 decision 2). It rides + # the new arm 3, which occupies exactly the pre-0153 arm-4 slot, so this is byte-identical. + cleartext_accepted=hop_insecure_escape_downgrades(enforcing=posture.enforcing), ) @@ -3876,8 +3913,40 @@ def _set(section: str, key: str, value: Any) -> None: _set("ai", "data_class", "phi" if sec.handles_real_patient_data else "synthetic") -def security_loosenings(sec: SecuritySettings) -> list[tuple[str, str]]: - """Every ``[security]`` switch currently at its INSECURE value, as ``(switch, plain-language risk)``. +def security_loosenings( + sec: SecuritySettings, + store: StoreSettings, + auth: AuthSettings, + cleartext_hops: Sequence[str], +) -> list[tuple[str, str]]: + """The ``[security]`` switches at their INSECURE value, plus the enumerated deviations outside that + section, as ``(switch, plain-language risk)``. + + **Scope, stated precisely so the gap is visible rather than implied.** This registry covers *every* + ``[security]`` switch — pinned by a completeness floor in ``tests/test_security_posture_defaults.py`` + that iterates ``SecuritySettings.model_fields`` and fails on an unreported, unexempted one — plus an + ENUMERATED set of deviations that live elsewhere: ``[store].aad_bind``, + ``[auth].ad_session_recheck_seconds``, and the per-connection ``cleartext_accepted``. It is NOT yet + an exhaustive registry of every security-relevant switch in every section; ``[store]``/``[auth]`` + carry others (``encrypt``, ``trust_server_certificate``, ``enabled``, ``require_mfa``, + ``ad_tls_verify``, ``ad_allow_insecure_ldap``, ``oidc_require_mfa_claim``, + ``password_check_breached``) that are gated elsewhere and are not reported here. That list is + enumerated in the floor test's exemption set so the gap is a written decision that a new switch + cannot silently join. + + Every parameter is REQUIRED, not optional, and deliberately so. There is exactly ONE shipped posture + and an operator may only loosen from it, so a deviation that this registry cannot see is a second + posture by the back door. An optional parameter is a detector that silently fails to fire; a required + one makes omission a type error at every call site. + + ``cleartext_hops`` is the list of CONNECTION NAMES that declare ``cleartext_accepted`` (ADR 0153) — + the one connection-scoped deviation in this otherwise settings-scoped registry. It arrives as plain + names rather than a ``Registry`` so ``config.settings`` never has to know the graph type; the caller + resolves them (``config.wiring.accepted_cleartext_hops`` is the shared reader, which walks both + outbound connections and ``FhirLookup`` read connections). A caller that genuinely has no graph — + ``messagefoundry security show``, which reads a settings file and never loads the connection config + — passes an empty sequence and SAYS SO in its output, rather than reporting a subset as if it were + everything. Shared by the serve-time loosening warning (``__main__``, ADR 0118 AC-4) and the read-only posture view (``GET /security/posture``, AC-5), so the two never drift. This is advisory only — it names what @@ -3995,6 +4064,40 @@ def security_loosenings(sec: SecuritySettings) -> list[tuple[str, str]]: ) if sec.allow_keeping_phi_indefinitely: out.append(("allow_keeping_phi_indefinitely", "unbounded PHI retention is permitted")) + # --- switches outside [security] that are still posture deviations (ADR 0148: one posture, loosen + # only). They live in [store]/[auth] for cohesion, but an operator turning either off is loosening the + # shipped posture, so they belong in the same registry rather than a parallel one that could drift. + if not store.aad_bind: + out.append( + ( + "aad_bind", + "at-rest values are NOT bound to their (table, column, row) cell — a ciphertext moved " + "between cells decrypts instead of failing its auth tag (no effect without a store key)", + ) + ) + # Conditional on ad_enabled, like allowed_client_networks above: with no directory there is nothing to + # reconcile against, so 0 is not a weaker choice, it is the only meaningful one. + if auth.ad_enabled and not auth.ad_session_recheck_seconds: + out.append( + ( + "ad_session_recheck_seconds", + "directory revocation does NOT propagate — an AD account disabled or deleted keeps its " + "live engine sessions until they expire on their own", + ) + ) + # --- the one CONNECTION-scoped deviation (ADR 0153 decision 2). It is not a [security] switch, but + # it is a declared departure from the one shipped posture, so it belongs in the one registry an + # operator reads — a deviation the registry cannot see is a second posture by the back door. + if cleartext_hops: + named = ", ".join(sorted(cleartext_hops)) + out.append( + ( + "cleartext_accepted", + f"{len(cleartext_hops)} connection(s) cross a CLEARTEXT hop by declaration " + f"({named}) — the payload, and any credential the connection carries, ride those hops " + "unencrypted and readable by anything on the path", + ) + ) return out diff --git a/messagefoundry/config/tls_policy.py b/messagefoundry/config/tls_policy.py index 5023a57f..d3d9a709 100644 --- a/messagefoundry/config/tls_policy.py +++ b/messagefoundry/config/tls_policy.py @@ -61,6 +61,7 @@ "TrustAnchorPolicy", "active_hop_posture", "build_verifying_client_context", + "cleartext_acceptance_audit_sink", "current_hop_posture", "enforce_insecure_hop", "harden_kex_groups", @@ -328,11 +329,12 @@ def _is_forward_secret(cipher: Mapping[str, object]) -> bool: class HopDisposition(enum.Enum): """What a cell must do with an insecure (cleartext / unverified-TLS) transport hop. - ``ALLOW`` — cross it silently (no PHI on the hop, a proven-loopback on-box hop, or an audited - attestation that the hop is secure by other means). ``WARN`` — cross it but log loudly + audit (a - non-production PHI hop, or a non-prod audited opt-out via the clamped global escape). ``REFUSE`` — - do not cross it: a production PHI hop with no attestation. Produced by - :func:`insecure_hop_disposition`; acted on by :func:`enforce_insecure_hop`.""" + ``ALLOW`` — cross it silently (a proven-loopback on-box hop, or an audited attestation that the hop + is secure by other means). ``WARN`` — cross it but log loudly + audit (an operator-declared + ``cleartext_accepted`` hop, or a non-enforcing instance). ``REFUSE`` — do not cross it: an enforcing + hop with neither an attestation nor an acceptance. Produced by :func:`insecure_hop_disposition`; + acted on by :func:`enforce_insecure_hop`. (ADR 0153: the data label no longer appears here — no + ``data_class`` value can ALLOW a cleartext hop.)""" ALLOW = "allow" WARN = "warn" @@ -398,43 +400,48 @@ def is_loopback_hop_host(host: str) -> bool: def insecure_hop_disposition( *, - is_phi: bool, enforcing: bool, is_loopback_hop: bool, hop_attested: bool, - audited_opt_out: bool, + cleartext_accepted: bool, ) -> HopDisposition: - """Decide what to do with an insecure transport hop, keyed on posture (#200, ADR 0092 — PURE). + """Decide what to do with a cleartext transport hop (#200, ADR 0092 — amended by ADR 0153; PURE). The single authority every insecure-egress cell consumes, so all decide identically. Explicit, early-return precedence (the order is load-bearing — the owner-ratified gradient): #. ``is_loopback_hop`` → :attr:`~HopDisposition.ALLOW` — an on-box hop is not a network exposure. #. ``hop_attested`` → :attr:`~HopDisposition.ALLOW` — a per-connection, load-validated, audited - attestation that this hop is legitimately secure (a proxy-terminated / trusted-segment hop). - #. not ``is_phi`` (synthetic instance) → :attr:`~HopDisposition.ALLOW` — no PHI rides the hop. - #. ``audited_opt_out`` → :attr:`~HopDisposition.WARN` — the global escape, **already clamped by the - caller** to non-enforcing (see ``settings.hop_insecure_escape_downgrades``); under ENFORCE the - caller passes ``False`` here, so this arm never fires for an enforcing PHI hop. - #. ``enforcing`` → :attr:`~HopDisposition.REFUSE` — an enforcing PHI hop with no attestation: do - not put PHI on the wire in the clear. - #. else (non-enforcing PHI — the WARN posture) → :attr:`~HopDisposition.WARN`. - - Note the escape (``audited_opt_out``) can never satisfy an enforcing PHI hop: it is clamped to - ``False`` under ENFORCE upstream, so the ``enforcing`` arm always wins there (decision 2). - Attestation (``hop_attested``) is the *only* per-hop way to cross an enforcing PHI hop, and it is + attestation that this hop *is* legitimately secure (a proxy-terminated / trusted-segment hop). + #. ``cleartext_accepted`` → :attr:`~HopDisposition.WARN` — the opposite claim (ADR 0153 decision + 2): this hop is **not** secure and the operator accepts that. Crossed, but logged at EVERY + construction and audited — an accepted risk that stops being visible has stopped being accepted. + #. not ``enforcing`` → :attr:`~HopDisposition.WARN` — the ``[security].enforcement`` dial of ADR + 0148, deliberately retained: it is an explicit operator dial (not a data-classification label) + and it still logs and audits every hop, so nothing goes silent. + #. else → :attr:`~HopDisposition.REFUSE` — an enforcing hop with neither an attestation nor an + acceptance: do not put the payload on the wire in the clear. + + **ADR 0153 removed two parameters.** ``is_phi`` is gone: no data label may permit a cleartext hop, + because ``data_class`` is authored in the same file as the hosts it governs and a typo in it is + indistinguishable from a declaration, with every transport hop in the product as its blast radius. + ``audited_opt_out`` is gone with it, so the blunt global ``MEFOR_ALLOW_INSECURE_TLS`` escape can no + longer influence a cleartext-hop decision (decision 5 — the variable survives for the six + non-connection cells that have no per-connection field to carry a declaration). + + Only an arm that returned ALLOW was deleted, so removing it can only ever turn a crossing into a + WARN or a REFUSE, never the reverse: ADR 0092 decision 5 (no-loosen) holds **by construction**. + ``hop_attested`` remains the only per-hop way to cross an enforcing hop *silently*, and it is audited when it suppresses a would-be enforcing refusal (the cell audits at that point).""" if is_loopback_hop: return HopDisposition.ALLOW if hop_attested: return HopDisposition.ALLOW - if not is_phi: - return HopDisposition.ALLOW - if audited_opt_out: + if cleartext_accepted: return HopDisposition.WARN - if enforcing: - return HopDisposition.REFUSE - return HopDisposition.WARN + if not enforcing: + return HopDisposition.WARN + return HopDisposition.REFUSE def enforce_insecure_hop( @@ -466,6 +473,46 @@ def enforce_insecure_hop( audit_sink(detail) +def cleartext_acceptance_audit_sink( + reason: str | None, *, connection: str | None = None +) -> Callable[[str], None]: + """The ``audit_sink`` :func:`enforce_insecure_hop` records an accepted cleartext hop through. + + ADR 0153 decision 2 requires a ``cleartext_accepted`` hop to be "logged at every construction and + recorded in the audit trail". This is the audit half: a distinct, structured record that names the + DECLARATION and its reason, so an auditor can tell an operator-accepted cleartext hop apart from a + hop that merely warned because the instance is not enforcing — the two produce the same + :attr:`~HopDisposition.WARN` and would otherwise be indistinguishable in the log. + + ``connection`` names the declaring connection. It is what makes the record ACTIONABLE: ``cell`` is a + static family label and the HTTP-family ``message`` carries a host but no port, so with two + destinations to the same host an auditor could otherwise not tell which declaration produced the + crossing — and the whole point of the record is to lead back to the line to fix. Every cell that can + know the name passes it; ``None`` renders as ``(unnamed)`` rather than a misleading blank. + + Built as a plain ``Callable`` so this stays a pure ``config``-level helper that never imports the + engine's ``AlertSink`` (one-way dependency boundary), and shared by BOTH insecure-hop guards (the + raw-transport one in ``transports.mllp`` and the HTTP-family one in ``transports.rest``) so the + record is never re-forked. ``reason`` is operator-authored text — it is logged verbatim as a + parameter, never interpolated into the format string, and the caller's ``detail`` carries only a + PHI-free cell label plus scheme/host/port, never a body or a credential. + + The marker is deliberately **lower-case**: the PHI redaction filter (``redaction._NAME_RUN``) treats + two or more adjacent ALL-CAPS tokens as a possible name run and replaces them with ``[redacted]``, + so a shouted marker would be scrubbed out of the very record it exists to make findable.""" + + def _record(detail: str) -> None: + logger.warning( + "cleartext hop crossed on an operator acceptance — connection %s: %s " + "(cleartext_accepted; reason: %s)", + connection or "(unnamed)", + detail, + reason or "(none provided)", + ) + + return _record + + #: The instance posture in force during connector construction. Stamped by the construction gate #: (``build_check_registry`` via :func:`active_hop_posture`) so a cell built inside that scope reads the #: LOADED config's derived posture rather than guessing. ``None`` when unstamped (an embedding/test that @@ -519,15 +566,27 @@ def phi_read_hop_disposition( authority's on-box carve-out (``is_loopback_hop``): a loopback / TLS / proxy-terminated serve hop is not an insecure network exposure, so PHI may cross (the serve-start exposed-gate already vetted it). There is no per-hop attestation for the API serve hop — the serve gate's proxy/TLS declarations are - what prove it secure — so ``hop_attested`` is always ``False`` here.""" + what prove it secure — so ``hop_attested`` is always ``False`` here. + + **ADR 0153 leaves this cell keyed on the data label, deliberately** (its *Explicitly out of scope* + table): the API serve hop is not a connection, so it has nowhere to carry a per-hop + ``cleartext_accepted`` declaration, and refusing it instead would create a deviation the loosening + registry cannot express. The ``not is_phi`` ALLOW arm 0153 deleted from the shared authority is + therefore restated HERE, explicitly, rather than inherited — so the scope limit is a written + decision at the one place it applies, not an accident of a signature. A ``[security]``-level + declaration for this cell is the recorded follow-up.""" if posture is None: return HopDisposition.ALLOW + if not posture.is_phi: + return HopDisposition.ALLOW return insecure_hop_disposition( - is_phi=posture.is_phi, enforcing=posture.enforcing, is_loopback_hop=serve_hop_secure, hop_attested=False, - audited_opt_out=audited_opt_out, + # The global escape keeps its arm HERE (same scope carve-out): it is the only expressible + # relaxation this non-connection cell has. Byte-identical to the pre-0153 arm 4 — WARN, ahead + # of the enforcing REFUSE — because arm 3 of the new precedence sits in exactly that slot. + cleartext_accepted=audited_opt_out, ) diff --git a/messagefoundry/config/wiring.py b/messagefoundry/config/wiring.py index 3928cb9e..c309e668 100644 --- a/messagefoundry/config/wiring.py +++ b/messagefoundry/config/wiring.py @@ -70,6 +70,7 @@ def handle(msg): Schedule, StallThreshold, Validation, + _check_cleartext_acceptance, ) from messagefoundry.config.send_snapshot import snapshot_on_send_active from messagefoundry.parsing.message import Message, RawMessage, snapshot_payload @@ -133,6 +134,7 @@ def handle(msg): "MessageTypeError", "load_config", "validate_config", + "accepted_cleartext_hops", ] _logger = logging.getLogger(__name__) @@ -496,6 +498,12 @@ def FhirLookup( timeout_seconds: float = 30.0, verify_tls: bool = True, # False (dev only) needs MEFOR_ALLOW_INSECURE_TLS encoding: str = "utf-8", + # ADR 0153 decision 2 — the same per-connection cleartext declaration an outbound carries. It must + # be authorable HERE: the read executor honours the pair, so leaving it to a hand-mutated + # `spec.settings` would be an escape with no load validation and nothing for the loosening registry + # to name — a deviation the registry cannot see is a second posture by the back door. + cleartext_accepted: bool = False, + cleartext_reason: str | None = None, ) -> FhirLookupSpec: """Declare a named live-lookup FHIR connection (ADR 0043). A Handler reads it at run time with ``fhir_lookup(name, query)`` — a **read-only** read-by-id (``"Patient/123"``) or search @@ -516,21 +524,38 @@ def FhirLookup( the engine only at allowed hosts. Put secrets (``bearer_token`` / ``basic_*`` / SMART keys) in :func:`env`. TLS is on by default; weakening it needs ``MEFOR_ALLOW_INSECURE_TLS``. The pure ``parsing/fhir/`` codec parses the reply, so a ``FhirLookup``-declaring graph needs the optional - ``messagefoundry[fhir]`` extra.""" - spec = FhirLookupSpec( - name, - { - "url": url, # stored under "url" (NOT base_url) so the egress gate reads the same key as FHIR() - "fhir_version": fhir_version, - "headers": headers or {}, - "bearer_token": bearer_token, - "basic_user": basic_user, - "basic_password": basic_password, - "timeout_seconds": timeout_seconds, - "verify_tls": verify_tls, - "encoding": encoding, - }, - ) + ``messagefoundry[fhir]`` extra. + + ``cleartext_accepted`` / ``cleartext_reason`` (ADR 0153) declare that this lookup's read hop is + cleartext, is not secure, and the operator accepts that — a mandatory written reason, a loud WARN + plus an audit record at every construction, and an entry in ``security_loosenings()`` / + ``GET /security/posture`` naming this connection. Same flag/reason coherence rules as an + ``outbound()``: the flag without a reason, a blank reason, or a reason without the flag all fail + loud at load.""" + # ADR 0153: coherence-checked at the ONE authoring surface, exactly as build_outbound_connection + # does for an outbound, so the declaration cannot reach the read executor unvalidated. + try: + _check_cleartext_acceptance(cleartext_accepted, cleartext_reason) + except ValueError as exc: + raise WiringError(f"fhir lookup {name!r}: {exc}") from exc + settings: dict[str, Any] = { + "url": url, # stored under "url" (NOT base_url) so the egress gate reads the same key as FHIR() + "fhir_version": fhir_version, + "headers": headers or {}, + "bearer_token": bearer_token, + "basic_user": basic_user, + "basic_password": basic_password, + "timeout_seconds": timeout_seconds, + "verify_tls": verify_tls, + "encoding": encoding, + } + if cleartext_accepted: + # Written only when declared, so an undeclared lookup's settings are byte-identical (and the + # redacted settings view, which several surfaces render, gains no empty governance keys). + settings["cleartext_accepted"] = True + settings["cleartext_reason"] = cleartext_reason + settings["cleartext_connection"] = name + spec = FhirLookupSpec(name, settings) _active_registry().add_fhir_lookup(spec) return spec @@ -2566,6 +2591,13 @@ class OutboundConnection: # around its ACK read). DISPLAY ONLY — no delivery effect, independent of `timeout_seconds`/pacing. # Default 0.0. Threaded to the Destination by _dest_config. Code-first AND connections.toml. waiting_display_delay: float = 0.0 + # ADR 0153 decision 2: this outbound's hop is cleartext, is NOT secure, and the operator accepts + # that (with a written reason). WARN, never ALLOW — crossed, but logged at every construction and + # audited. The opposite claim to `tls_hop_attested` ("this hop IS secure by means the engine cannot + # see"), deliberately kept a separate field so the audit trail can tell a proxy-terminated hop from + # plaintext on a flat network. Outbound-only. Threaded to the Destination by _dest_config. + cleartext_accepted: bool = False + cleartext_reason: str | None = None source_file: str | None = None source_line: int | None = None @@ -2889,6 +2921,35 @@ def _active_registry() -> Registry: return _active +def accepted_cleartext_hops(registry: Registry) -> list[tuple[str, str]]: + """Every connection that DECLARES ``cleartext_accepted``, as ``(name, reason)`` (ADR 0153). + + The SINGLE reader of the accepted set, shared by ``messagefoundry check``'s ``cleartext-accepted`` + surface and by the API's ``GET /security/posture`` loosening registry, so the two can never report + different sets. Sorted by connection name for a stable, diffable list. + + It walks **both** connection tables that can cross a declared cleartext hop: ``outbound`` and + ``fhir_lookups``. The lookups are not optional coverage — the read executor honours the declaration + for a PHI-bearing read hop, so omitting them here would let a live cleartext hop cross while every + visibility surface reported the accepted set as empty. Lookup names are prefixed ``fhir_lookup:`` + because they live in a separate namespace and could otherwise collide with an outbound's name. + + Pure — it reads the loaded graph and touches nothing else. It lives HERE, beside the ``Registry`` it + reads, rather than in ``checks`` or ``api``, so neither of those has to import the other: the + acceptance is connection-scoped by construction, which is exactly why the settings-scoped + ``security_loosenings`` takes the resolved NAMES rather than a graph.""" + out = [ + (oc.name, oc.cleartext_reason or "(none recorded)") + for oc in registry.outbound.values() + if oc.cleartext_accepted + ] + for spec in registry.fhir_lookups.values(): + if spec.settings.get("cleartext_accepted"): + reason = spec.settings.get("cleartext_reason") + out.append((f"fhir_lookup:{spec.name}", str(reason) if reason else "(none recorded)")) + return sorted(out) + + def _call_site() -> tuple[str | None, int | None]: """File + line of the config module that called the declaration (for IDE go-to-definition).""" caller = sys._getframe(2) # _call_site -> inbound/outbound -> config module @@ -3322,6 +3383,8 @@ def build_outbound_connection( metadata: Mapping[str, Any] | None = None, flagged: bool = False, waiting_display_delay: float = 0.0, + cleartext_accepted: bool = False, + cleartext_reason: str | None = None, source_file: str | None = None, source_line: int | None = None, ) -> OutboundConnection: @@ -3329,6 +3392,15 @@ def build_outbound_connection( The shared core of code-first :func:`outbound` **and** the ``connections.toml`` loader (ADR 0007). Pure — it does not touch the active registry; the caller is responsible for ``add_outbound``.""" + # ADR 0153 decision 2: the cleartext-acceptance pair is coherence-checked HERE, the one choke point + # both authoring surfaces pass through, so a flag with no reason (or a reason with no flag) fails at + # `messagefoundry check` / dry-run rather than at connector construction. Re-raised as a WiringError + # so it surfaces with the connection name attached, exactly like every other wiring invariant; the + # Destination model re-validates it independently (defense in depth for a hand-built Destination). + try: + _check_cleartext_acceptance(cleartext_accepted, cleartext_reason) + except ValueError as exc: + raise WiringError(f"outbound connection {name!r}: {exc}") from exc if dead_letter_days is not None and dead_letter_days < 0: # Per-connection dead-letter retention override (#34, ADR 0027). None = inherit # [retention].dead_letter_days; 0 = keep forever; >0 = days. A negative window is meaningless — @@ -3521,6 +3593,8 @@ def build_outbound_connection( metadata=metadata, flagged=flagged, waiting_display_delay=waiting_display_delay, + cleartext_accepted=cleartext_accepted, + cleartext_reason=cleartext_reason, source_file=source_file, source_line=source_line, ) @@ -3545,6 +3619,8 @@ def outbound( metadata: Mapping[str, Any] | None = None, flagged: bool = False, waiting_display_delay: float = 0.0, + cleartext_accepted: bool = False, + cleartext_reason: str | None = None, ) -> None: """Declare an outbound connection that Handlers can ``Send`` to. @@ -3572,7 +3648,16 @@ def outbound( builds it, never resolves its ``env()`` values and spawns no delivery worker — a ``Send`` to it is declined and logged rather than queued. Unlike ``simulate=True`` (the lane IS built and DOES take rows) and unlike a DR/scheduler park (rows are retained and retried), **nothing queues to it at - all**. It **wins** over ``auto_start``. Also a ``connections.toml`` key.""" + all**. It **wins** over ``auto_start``. Also a ``connections.toml`` key. + + ``cleartext_accepted`` (ADR 0153) declares that **this** outbound's hop is cleartext, is NOT secure, + and the operator accepts that — with a mandatory ``cleartext_reason`` recorded for the audit trail. + It yields a loud WARN at every construction (never a silent ALLOW) and lets the hop cross even under + ``[security].enforcement = enforce``. It is the opposite claim to a connection's ``tls_hop_attested`` + ("this hop *is* secure by means the engine cannot see", which ALLOWs), and the two are deliberately + separate so the audit trail can tell a proxy-terminated hop from plaintext on a flat network. For + ``Tcp()``/``X12()``, which have no TLS support at all, it is a **permanent, structural** declaration + — there is no ``tls = true`` for them to migrate to (BACKLOG #311). Also a ``connections.toml`` key.""" file, line = _call_site() _active_registry().add_outbound( build_outbound_connection( @@ -3593,6 +3678,8 @@ def outbound( metadata=metadata, flagged=flagged, waiting_display_delay=waiting_display_delay, + cleartext_accepted=cleartext_accepted, + cleartext_reason=cleartext_reason, source_file=file, source_line=line, ) diff --git a/messagefoundry/pipeline/wiring_runner.py b/messagefoundry/pipeline/wiring_runner.py index 77c51460..d79befbc 100644 --- a/messagefoundry/pipeline/wiring_runner.py +++ b/messagefoundry/pipeline/wiring_runner.py @@ -5688,6 +5688,17 @@ def _dest_config( # ADR 0126: merge the site-wide forward-proxy default (a per-connection proxy wins). This is the one # choke point feeding start/check/dry-run, so the same effective proxy is built at all three. _apply_egress_proxy_default(settings, egress) + # ADR 0153: MIRROR the cleartext-acceptance declaration into the resolved settings. The connectors + # read the typed Destination fields below, but the deep settings-driven seams — the forward-proxy + # credential chain, the HTTP Digest / OAuth2 / SMART token-endpoint providers — receive only a + # settings mapping, exactly as they already do for `tls_hop_attested`. The connection NAME rides + # with it so the acceptance audit record those seams emit can still name the declaration that + # produced it. Written ONLY when the flag is set, so an outbound that declared nothing carries no + # new keys and is byte-identical. + if oc.cleartext_accepted: + settings["cleartext_accepted"] = True + settings["cleartext_reason"] = oc.cleartext_reason + settings["cleartext_connection"] = oc.name return Destination( name=oc.name, type=oc.spec.type, @@ -5702,6 +5713,13 @@ def _dest_config( # legitimately-secure egress hop even on production-PHI. Default False → keyed purely on posture. tls_hop_attested=bool(settings.get("tls_hop_attested", False)), tls_hop_attested_reason=_hop_attested_reason(settings), + # ADR 0153 decision 2: the per-outbound cleartext-hop ACCEPTANCE ("this hop is NOT secure and we + # accept that"). A TOP-LEVEL outbound key, not a transport setting, so it is read off the + # OutboundConnection rather than the env-resolved settings dict — one authoring surface, and no + # env() indirection on a governance declaration that must be legible in review. Default off → + # byte-identical. + cleartext_accepted=oc.cleartext_accepted, + cleartext_reason=oc.cleartext_reason, # #201 (ADR 0078 amendment): per-connection attestation that revocation is checked for a VERIFYING # outbound TLS hop, typed here so the connector's revocation gate can ALLOW it even on prod-PHI. # Default False → keyed purely on posture (existing verifying outbounds byte-identical). diff --git a/messagefoundry/store/base.py b/messagefoundry/store/base.py index 1b9c52d0..108727dd 100644 --- a/messagefoundry/store/base.py +++ b/messagefoundry/store/base.py @@ -1760,7 +1760,8 @@ async def open_store( """ # The at-rest cipher via the single build_store_cipher seam: ADR 0019 key sourcing + the ADR 0138 # cipher_provider dispatch. Default `aesgcm` is the in-process AES-256-GCM keyring (active + retired - # decrypt-only, write_v2=aad_bind), byte-identical at rest (CRYPTO-1). `vault_transit` runs the bulk + # decrypt-only, write_v2=aad_bind — which now defaults ON, so new writes are cell-bound mfenc:v2; + # aad_bind=false selects the frozen v1 writer, byte-identical at rest). `vault_transit` runs the bulk # crypto inside Vault/OpenBao Transit so the DEK never enters heap (ASVS 13.3.3). No key → identity. cipher = build_store_cipher(settings) # #190: HKDF-derived HMAC key for the tamper-evident audit chain; None for the identity cipher (the diff --git a/messagefoundry/store/crypto.py b/messagefoundry/store/crypto.py index 14011bac..a60618b4 100644 --- a/messagefoundry/store/crypto.py +++ b/messagefoundry/store/crypto.py @@ -32,8 +32,10 @@ cryptographically bound to the cell it lives in — a blob cut-and-pasted into another row/column/table fails the auth tag (``CipherError``) instead of decrypting. ``v1`` never carries AAD and stays frozen; :meth:`~AesGcmCipher.decrypt` dispatches the AAD by marker version (v1→``None``, v2→caller-supplied), so -legacy v1 rows still read (dual-read). Bound writes are opt-in via ``[store].aad_bind`` (which sets -``write_v2``); off by default, so the default at-rest format is unchanged (CRYPTO-1). +legacy v1 rows still read (dual-read). Bound writes are selected by ``[store].aad_bind`` (which sets +``write_v2``); **ON by default** since ADR 0148 GIVEN 1, so the default at-rest format is ``mfenc:v2``. +Setting the knob false selects the frozen v1 writer — byte-identical at rest (CRYPTO-1), and a declared +loosening. **Key rotation (WP-5, ASVS 11.2.2).** The cipher is a **keyring**: it encrypts with the single *active* key and decrypts with whichever configured key matches (active + any retired decrypt-only diff --git a/messagefoundry/store/postgres.py b/messagefoundry/store/postgres.py index c36ac3c8..36713838 100644 --- a/messagefoundry/store/postgres.py +++ b/messagefoundry/store/postgres.py @@ -1124,8 +1124,9 @@ async def require_rcsi_for_pooled(self) -> None: # --- PHI-at-rest cipher seam for nullable text columns (WP-5) ------------- # Cell-bound AAD (ASVS 11.3.3, ADR 0019): `aad` is REQUIRED so mypy-strict flags any un-threaded - # site; the caller passes cell_aad(table, column, *pk). Ignored while aad_bind is off (v1 writer + - # v1 dual-read), so passing it is always safe. Mirrors MessageStore._enc/_dec. + # site; the caller passes cell_aad(table, column, *pk). Bound on the shipped aad_bind=true default; + # ignored when an operator sets aad_bind false (v1 writer + v1 dual-read), so passing it is always + # safe either way. Mirrors MessageStore._enc/_dec. def _enc(self, value: str | None, *, aad: bytes) -> str | None: if not value: # None or "" → leave blank (covers purged/empty values) diff --git a/messagefoundry/store/sqlserver.py b/messagefoundry/store/sqlserver.py index adf68c39..03cca7c7 100644 --- a/messagefoundry/store/sqlserver.py +++ b/messagefoundry/store/sqlserver.py @@ -1949,8 +1949,9 @@ async def _close_claim_holders(self) -> None: # '' through unchanged on read (so a no-key -> key restart reads pre-existing plaintext correctly). # Cell-bound AAD (ASVS 11.3.3, ADR 0019): `aad` is REQUIRED so mypy-strict flags any un-threaded - # site; the caller passes cell_aad(table, column, *pk). Ignored while aad_bind is off (v1 writer + - # v1 dual-read), so passing it is always safe. Mirrors MessageStore._enc/_dec. + # site; the caller passes cell_aad(table, column, *pk). Bound on the shipped aad_bind=true default; + # ignored when an operator sets aad_bind false (v1 writer + v1 dual-read), so passing it is always + # safe either way. Mirrors MessageStore._enc/_dec. def _enc(self, value: str | None, *, aad: bytes) -> str | None: if not value: # None or "" → leave blank (covers purged/empty values) diff --git a/messagefoundry/store/store.py b/messagefoundry/store/store.py index 4430d531..b76d409a 100644 --- a/messagefoundry/store/store.py +++ b/messagefoundry/store/store.py @@ -1800,8 +1800,9 @@ def __init__( # Cell-bound AAD (ASVS 11.3.3, ADR 0019): `aad` is a REQUIRED keyword — the caller passes # cell_aad(table, column, *pk) for the exact cell being written/read, so a v2 ciphertext is bound to # its (table, column, row). Making it required (not defaulted) makes mypy-strict flag any un-threaded - # site — a half-threaded cell would fail closed under aad_bind. With aad_bind OFF the cipher writes v1 - # and ignores `aad` (dual-read of every legacy row), so passing it is always safe. + # site — a half-threaded cell fails closed under the shipped aad_bind=true default. With aad_bind + # false the cipher writes v1 and ignores `aad` (dual-read of every legacy row), so passing it is + # always safe either way. def _enc(self, value: str | None, *, aad: bytes) -> str | None: if not value: # None or "" → leave blank (covers purged/empty values) diff --git a/messagefoundry/transports/dicom.py b/messagefoundry/transports/dicom.py index 9a089793..abf017a8 100644 --- a/messagefoundry/transports/dicom.py +++ b/messagefoundry/transports/dicom.py @@ -506,6 +506,12 @@ def __init__(self, config: Destination) -> None: description="plaintext DIMSE C-STORE association", attested=config.tls_hop_attested, attested_reason=config.tls_hop_attested_reason, + # ADR 0153: `cleartext_accepted` crosses this hop with a loud, audited WARN. Unlike + # Tcp()/X12() it is TRANSITIONAL here — DICOM() supports tls=true, so the declaration + # should end when the peer does. + cleartext_accepted=config.cleartext_accepted, + cleartext_reason=config.cleartext_reason, + connection=config.name, ) if self._ssl is None else None diff --git a/messagefoundry/transports/dicomweb.py b/messagefoundry/transports/dicomweb.py index 7bc24d0c..7761b7b3 100644 --- a/messagefoundry/transports/dicomweb.py +++ b/messagefoundry/transports/dicomweb.py @@ -157,11 +157,19 @@ def __init__(self, config: Destination) -> None: self.capture_response: bool = bool(s.get("capture_response", False)) # #200 (ADR 0092): the per-connection insecure-hop attestation, keying the posture-keyed refusal. attested = config.tls_hop_attested + # ADR 0153 decision 2: the OPPOSITE per-connection declaration — this hop is cleartext, is not + # secure, and that is accepted (WARN + audit, never a silent ALLOW). + accepted, accept_reason = config.cleartext_accepted, config.cleartext_reason # Captured at construction; re-asserted (zero I/O) at the byte-crossing in _post (decision 4). self._hop_guard: InsecureHopGuard | None = None # BACKLOG #112/#127/#128 (ADR 0126): per-connection forward/egress proxy (None → byte-identical). self._proxy: ProxyConfig | None = egress_route_from_settings( - s, dest_scheme=scheme, attested=attested + s, + dest_scheme=scheme, + attested=attested, + cleartext_accepted=accepted, + cleartext_reason=accept_reason, + connection=config.name, ) dest_host = urllib.parse.urlsplit(self.base_url).hostname or "" proxy_dest = self._proxy.for_host(dest_host) if self._proxy is not None else None @@ -173,10 +181,25 @@ def __init__(self, config: Destination) -> None: # The multipart Content-Type (with the generated boundary) is set per-request in _post — it is not # operator-supplied, so it is excluded from the length check (which guards URL + supplied headers). enforce_outbound_length_limits(self.base_url, self._headers) - refuse_cleartext_credentials(scheme, self._headers, self.base_url, attested=attested) + refuse_cleartext_credentials( + scheme, + self._headers, + self.base_url, + attested=attested, + cleartext_accepted=accepted, + cleartext_reason=accept_reason, + connection=config.name, + ) # ASVS 12.2.1: the STOW-RS multipart body carries the DICOM object (PHI), so a cleartext http # egress to a non-loopback host is refused even without credentials (loopback byte-identical). - self._hop_guard = refuse_cleartext_egress(scheme, self.base_url, attested=attested) + self._hop_guard = refuse_cleartext_egress( + scheme, + self.base_url, + attested=attested, + cleartext_accepted=accepted, + cleartext_reason=accept_reason, + connection=config.name, + ) if bool(s.get("verify_tls", True)): # #201 (ADR 0078 amendment): the verify-ON https hop validates the DICOMweb-server cert but # does no OCSP/CRL revocation (stdlib ssl has none) — refuse an off-loopback production-PHI diff --git a/messagefoundry/transports/email.py b/messagefoundry/transports/email.py index 08b79ec9..0a47cff7 100644 --- a/messagefoundry/transports/email.py +++ b/messagefoundry/transports/email.py @@ -124,13 +124,23 @@ def __init__(self, config: Destination) -> None: # process-wide env var — without this, attestation would be dead config on EMAIL alone. self._hop_guard: InsecureHopGuard | None = None if not self.use_tls: - if not (weakened_tls_escape_permitted_here() or config.tls_hop_attested): + # ADR 0153: `cleartext_accepted` joins this disjunction. The pre-gate fires BEFORE the shared + # authority below, so without this arm an SMTP outbound that declared the acceptance would + # still be refused here and the declaration would be dead config on EMAIL alone — exactly the + # trap `tls_hop_attested` was added to this gate to avoid. Passing the pre-gate is not + # permission to cross: the guard below still decides the hop, and an accepted hop is WARNed + # and audited there, never silently allowed. + if not ( + weakened_tls_escape_permitted_here() + or config.tls_hop_attested + or config.cleartext_accepted + ): raise ValueError( "Email destination use_tls=false sends the message (and any credentials) over " f"cleartext SMTP; refused unless {INSECURE_TLS_ESCAPE_ENV} is set " - "(dev/trusted-network only) or the connection sets tls_hop_attested=true — use " - "STARTTLS (the default). Refused on a production-PHI instance even with the " - "escape (#200)." + "(dev/trusted-network only), or the connection sets tls_hop_attested=true (the hop " + "IS secure by other means) or cleartext_accepted=true with a cleartext_reason (the " + "hop is NOT secure and that is accepted) — use STARTTLS (the default)." ) # Credentials over an un-encrypted channel are never allowed, even with the escape: a # cleartext AUTH puts the password on the wire (the refuse_cleartext_credentials rule). @@ -154,6 +164,9 @@ def __init__(self, config: Destination) -> None: description="cleartext SMTP egress (use_tls=false)", attested=config.tls_hop_attested, attested_reason=config.tls_hop_attested_reason, + cleartext_accepted=config.cleartext_accepted, + cleartext_reason=config.cleartext_reason, + connection=config.name, ) self._hop_guard.enforce_construction() else: diff --git a/messagefoundry/transports/fhir.py b/messagefoundry/transports/fhir.py index 16d33542..7813790c 100644 --- a/messagefoundry/transports/fhir.py +++ b/messagefoundry/transports/fhir.py @@ -69,6 +69,7 @@ _NoRedirectHandler, _redact_url, capture_response_headers, + cleartext_acceptance_from_settings, egress_route_from_settings, enforce_outbound_length_limits, normalize_header_allowlist, @@ -257,11 +258,19 @@ def __init__(self, config: Destination) -> None: self.consumes_metadata: bool = bool(s.get("dynamic_headers", False)) # #200 (ADR 0092): the per-connection insecure-hop attestation, keying the posture-keyed refusal. attested = config.tls_hop_attested + # ADR 0153 decision 2: the OPPOSITE per-connection declaration — this hop is cleartext, is not + # secure, and that is accepted (WARN + audit, never a silent ALLOW). + accepted, accept_reason = config.cleartext_accepted, config.cleartext_reason # Captured at construction; re-asserted (zero I/O) at the byte-crossing in _post (decision 4). self._hop_guard: InsecureHopGuard | None = None # BACKLOG #112/#127/#128 (ADR 0126): per-connection forward/egress proxy (None → byte-identical). self._proxy: ProxyConfig | None = egress_route_from_settings( - s, dest_scheme=scheme, attested=attested + s, + dest_scheme=scheme, + attested=attested, + cleartext_accepted=accepted, + cleartext_reason=accept_reason, + connection=config.name, ) dest_host = urllib.parse.urlsplit(self.base_url).hostname or "" proxy_dest = self._proxy.for_host(dest_host) if self._proxy is not None else None @@ -272,10 +281,25 @@ def __init__(self, config: Destination) -> None: # Pre-emptive Proxy-Authorization (Basic; empty for Digest/none) — tunnelled for https (0126). self._headers.update(proxy_dest.auth_headers()) enforce_outbound_length_limits(self.base_url, self._headers) - refuse_cleartext_credentials(scheme, self._headers, self.base_url, attested=attested) + refuse_cleartext_credentials( + scheme, + self._headers, + self.base_url, + attested=attested, + cleartext_accepted=accepted, + cleartext_reason=accept_reason, + connection=config.name, + ) # ASVS 12.2.1: the FHIR resource/Bundle body is PHI, so a cleartext http egress to a # non-loopback host is refused even without credentials (loopback stays byte-identical). - self._hop_guard = refuse_cleartext_egress(scheme, self.base_url, attested=attested) + self._hop_guard = refuse_cleartext_egress( + scheme, + self.base_url, + attested=attested, + cleartext_accepted=accepted, + cleartext_reason=accept_reason, + connection=config.name, + ) # ASVS 4.1.5 (ADR 0018): opt-in detached-JWS signing; None = off (byte-identical). Built here so # a bad key fails loud at construction; the signature is minted in _post over the body bytes. self._signer: MessageSigner | None = signer_from_destination(config) @@ -296,6 +320,9 @@ def __init__(self, config: Destination) -> None: {**self._headers, "Authorization": "Bearer"}, self.base_url, attested=attested, + cleartext_accepted=accepted, + cleartext_reason=accept_reason, + connection=config.name, ) if bool(s.get("verify_tls", True)): @@ -726,9 +753,20 @@ def __init__( self._encoding[cname] = str(s.get("encoding", "utf-8")) # #200 (ADR 0092): the per-connection insecure-hop attestation keys the posture-keyed refusal. attested = bool(s.get("tls_hop_attested", False)) + # ADR 0153: a FhirLookup connection has no Destination, so its cleartext-acceptance pair + # rides the spec settings, written there by the FhirLookup() factory (which load-validates + # the flag/reason coherence, exactly as build_outbound_connection does for an outbound). + lk_accepted, lk_reason, _ = cleartext_acceptance_from_settings(s) # BACKLOG #112/#127/#128 (ADR 0126): per-connection forward/egress proxy for the read hop AND # the SMART token endpoint (None → byte-identical). Bypass resolved per target host (#128). - proxy = egress_route_from_settings(s, dest_scheme=scheme, attested=attested) + proxy = egress_route_from_settings( + s, + dest_scheme=scheme, + attested=attested, + cleartext_accepted=lk_accepted, + cleartext_reason=lk_reason, + connection=cname, + ) base_host = urllib.parse.urlsplit(url).hostname or "" proxy_dest = proxy.for_host(base_host) if proxy is not None else None proxy_handlers = proxy_dest.opener_handlers() if proxy_dest is not None else () @@ -738,10 +776,25 @@ def __init__( # The read sends Authorization (static or SMART) — refuse it over cleartext http. token = token_provider_from_settings(s, proxy=proxy) check_headers = {**headers, "Authorization": "Bearer"} if token is not None else headers - refuse_cleartext_credentials(scheme, check_headers, url, attested=attested) + refuse_cleartext_credentials( + scheme, + check_headers, + url, + attested=attested, + cleartext_accepted=lk_accepted, + cleartext_reason=lk_reason, + connection=cname, + ) # ASVS 12.2.1: a cleartext read pulls the PHI resource/searchset back over the wire, so a # cleartext http read to a non-loopback host is refused too (loopback stays byte-identical). - self._hop_guard[cname] = refuse_cleartext_egress(scheme, url, attested=attested) + self._hop_guard[cname] = refuse_cleartext_egress( + scheme, + url, + attested=attested, + cleartext_accepted=lk_accepted, + cleartext_reason=lk_reason, + connection=cname, + ) self._headers[cname] = headers self._token[cname] = token if bool(s.get("verify_tls", True)): diff --git a/messagefoundry/transports/http_auth.py b/messagefoundry/transports/http_auth.py index 92a9c773..e812df80 100644 --- a/messagefoundry/transports/http_auth.py +++ b/messagefoundry/transports/http_auth.py @@ -51,6 +51,7 @@ ProxyConfig, _no_redirect_opener, _redact_url, + cleartext_acceptance_from_settings, proxy_auth_handler_from_settings, refuse_cleartext_credential_hop, ) @@ -125,6 +126,9 @@ def __init__( expiry_skew_seconds: float = _DEFAULT_EXPIRY_SKEW, timeout_seconds: float = _DEFAULT_TOKEN_TIMEOUT, attested: bool = False, + cleartext_accepted: bool = False, + cleartext_reason: str | None = None, + connection: str | None = None, proxy: ProxyConfig | None = None, ) -> None: if not token_url: @@ -146,13 +150,23 @@ def __init__( # subclasses, so the loader surfaces either identically. The message never carries the secret. try: refuse_cleartext_credential_hop( - scheme, token_url, credential="OAuth2 client_secret", attested=attested + scheme, + token_url, + credential="OAuth2 client_secret", + attested=attested, + # ADR 0153: the same per-connection declaration the delivery hop carries. Without it the + # token-endpoint hop would refuse a connection whose delivery hop was declared, leaving + # the operator no honest way to describe a legacy peer (see the note on + # refuse_cleartext_credential_hop). + cleartext_accepted=cleartext_accepted, + cleartext_reason=cleartext_reason, + connection=connection, ) except InsecureHopRefused as exc: raise HttpAuthError( "OAuth2 token endpoint over cleartext http would expose the client_secret; refused by " - "the instance security posture — a production-PHI hop cannot be escaped (use https, or " - "attest the hop as secure via tls_hop_attested)" + "the instance security posture (use https, attest the hop as secure via " + "tls_hop_attested, or declare cleartext_accepted with a cleartext_reason)" ) from exc if not client_id: raise HttpAuthError("OAuth2 client-credentials requires an 'oauth2_client_id' setting") @@ -278,6 +292,7 @@ def oauth2_cc_provider_from_settings( return None if not s.get("oauth2_enabled", True): return None + _accepted = cleartext_acceptance_from_settings(s) return OAuth2ClientCredentialsProvider( token_url=str(s.get("oauth2_token_url") or ""), client_id=str(s.get("oauth2_client_id") or ""), @@ -291,6 +306,12 @@ def oauth2_cc_provider_from_settings( # __init__ (read from settings exactly as _dest_config / FhirLookup do). Default False → the hop # decides purely on posture. attested=bool(s.get("tls_hop_attested", False)), + # ADR 0153: the sibling cleartext-acceptance declaration, mirrored into these resolved settings + # by the runner's _dest_config for exactly this kind of settings-driven seam (the connection name + # rides with it so the acceptance audit record can name the declaration that produced it). + cleartext_accepted=_accepted[0], + cleartext_reason=_accepted[1], + connection=_accepted[2], proxy=proxy, # ADR 0126: forward-proxy the token-endpoint POST ) @@ -343,15 +364,24 @@ def digest_handler_from_settings( # (both are ``ValueError``s → the loader surfaces either identically). Runs at connector construction # under the gate's stamped posture (fail-closing to prod-PHI when unstamped). attested = bool(s.get("tls_hop_attested", False)) + # ADR 0153: the sibling cleartext-acceptance declaration, mirrored into these resolved settings by + # the runner's _dest_config for exactly this kind of settings-driven seam. + accepted, accept_reason, accept_conn = cleartext_acceptance_from_settings(s) try: refuse_cleartext_credential_hop( - scheme, url, credential="digest credential", attested=attested + scheme, + url, + credential="digest credential", + attested=attested, + cleartext_accepted=accepted, + cleartext_reason=accept_reason, + connection=accept_conn, ) except InsecureHopRefused as exc: raise HttpAuthError( "HTTP Digest over cleartext http would expose the digest credential; refused by the " - "instance security posture — a production-PHI hop cannot be escaped (use https, or attest " - "the hop as secure via tls_hop_attested)" + "instance security posture (use https, attest the hop as secure via tls_hop_attested, or " + "declare cleartext_accepted with a cleartext_reason)" ) from exc user = str(s.get("http_auth_user") or "") password = str(s.get("http_auth_password") or "") diff --git a/messagefoundry/transports/mllp.py b/messagefoundry/transports/mllp.py index 6704af6a..a34e1215 100644 --- a/messagefoundry/transports/mllp.py +++ b/messagefoundry/transports/mllp.py @@ -36,7 +36,6 @@ from messagefoundry.config.models import AckMode, ConnectorType, Destination, Source from messagefoundry.config.settings import ( INSECURE_TLS_ESCAPE_ENV, - hop_insecure_escape_downgrades, weakened_tls_escape_permitted_here, ) from messagefoundry.config.tls_policy import ( @@ -46,6 +45,7 @@ RevocationHopGuard, TrustAnchorPolicy, build_verifying_client_context, + cleartext_acceptance_audit_sink, current_hop_posture, enforce_insecure_hop, harden_kex_groups, @@ -142,6 +142,15 @@ class InsecureHopGuard: attested: bool attested_reason: str | None posture: HopPosture | None + # ADR 0153 decision 2: the operator's declaration that THIS hop is cleartext, is not secure, and + # that is accepted. Distinct from `attested` above, which claims the opposite (the hop IS secure by + # means the engine cannot see) — never fuse the two, the audit trail exists to tell them apart. + cleartext_accepted: bool = False + cleartext_reason: str | None = None + # The DECLARING connection's name. It is what makes the acceptance audit record actionable: `cell` + # is a static family label, so with two outbounds to the same host an auditor could otherwise not + # tell which declaration produced the crossing. + connection: str | None = None @classmethod def capture( @@ -153,6 +162,9 @@ def capture( description: str, attested: bool, attested_reason: str | None, + cleartext_accepted: bool = False, + cleartext_reason: str | None = None, + connection: str | None = None, ) -> InsecureHopGuard: """Snapshot the decision inputs + the active hop posture for a cleartext outbound hop. ``cell`` is a short PHI-free label of the crossing; ``description`` explains the hop (scheme only — never a @@ -164,18 +176,21 @@ def capture( description=description, attested=attested, attested_reason=attested_reason, + cleartext_accepted=cleartext_accepted, + cleartext_reason=cleartext_reason, + connection=connection, posture=current_hop_posture(), ) def _disposition(self, posture: HopPosture) -> HopDisposition: return insecure_hop_disposition( - is_phi=posture.is_phi, enforcing=posture.enforcing, is_loopback_hop=is_loopback_hop_host(self.host), hop_attested=self.attested, - # The global escape is CLAMPED to non-enforcing upstream (settings.hop_insecure_escape_ - # downgrades), so under ENFORCE this is always False and can never satisfy an enforcing PHI hop. - audited_opt_out=hop_insecure_escape_downgrades(enforcing=posture.enforcing), + # ADR 0153: the data label is gone, and with it the blunt global MEFOR_ALLOW_INSECURE_TLS + # escape — a cleartext hop is now crossed only on-box, on an attestation, on a per-connection + # acceptance, or under a non-enforcing dial. + cleartext_accepted=self.cleartext_accepted, ) def _detail(self) -> str: @@ -183,19 +198,22 @@ def _detail(self) -> str: def enforce_construction(self) -> None: """The ENFORCED construction gate: raise - :class:`~messagefoundry.config.tls_policy.InsecureHopRefused` on a production-PHI cleartext hop, - loud-log (+ audit the attestation) on a warned hop, allow the rest. No-op when the posture is - unstamped (``None``) — the build_check gate is the authority; see the class docstring.""" + :class:`~messagefoundry.config.tls_policy.InsecureHopRefused` on an unattested, unaccepted + enforcing cleartext hop, loud-log (+ audit the attestation / the acceptance) on a warned hop, + allow the rest. No-op when the posture is unstamped (``None``) — the build_check gate is the + authority; see the class docstring.""" posture = self.posture if posture is None: return disposition = self._disposition(posture) - # Audit an attestation that SUPPRESSED a would-be production-PHI refusal (decision 3): the - # disposition is ALLOW only because `tls_hop_attested` fired before the production REFUSE arm. + # Audit an attestation that SUPPRESSED a would-be enforcing refusal (ADR 0092 decision 3): the + # disposition is ALLOW only because `tls_hop_attested` fired before the REFUSE arm. The + # `posture.is_phi` conjunct this branch used to carry went with ADR 0153 — the authority no + # longer reads the label, so gating the audit on it would silence the record for exactly the + # hops that newly depend on the attestation. if ( disposition is HopDisposition.ALLOW and self.attested - and posture.is_phi and posture.enforcing and not is_loopback_hop_host(self.host) ): @@ -205,7 +223,20 @@ def enforce_construction(self) -> None: self._detail(), self.attested_reason or "(none provided)", ) - enforce_insecure_hop(disposition, message=self._detail(), cell=self.cell) + enforce_insecure_hop( + disposition, + message=self._detail(), + cell=self.cell, + # ADR 0153 decision 2: an ACCEPTED cleartext hop is recorded at EVERY construction, not just + # warned — an accepted risk that stops being visible has stopped being accepted. Only wired + # when the acceptance is what produced the WARN, so a merely non-enforcing instance does not + # manufacture acceptance records for hops nobody declared. + audit_sink=( + cleartext_acceptance_audit_sink(self.cleartext_reason, connection=self.connection) + if disposition is HopDisposition.WARN and self.cleartext_accepted + else None + ), + ) def assert_send(self) -> None: """Zero-I/O send-time backstop: re-assert the captured decision at the byte crossing (defense in @@ -681,6 +712,11 @@ def __init__(self, config: Destination) -> None: description="cleartext MLLP egress", attested=config.tls_hop_attested, attested_reason=config.tls_hop_attested_reason, + # ADR 0153: `cleartext_accepted` crosses this hop with a loud, audited WARN. TRANSITIONAL + # here — MLLP() supports tls=true, so the declaration should end when the peer does. + cleartext_accepted=config.cleartext_accepted, + cleartext_reason=config.cleartext_reason, + connection=config.name, ) if self._ssl is None else None diff --git a/messagefoundry/transports/remotefile.py b/messagefoundry/transports/remotefile.py index 8866f352..0b1b5d25 100644 --- a/messagefoundry/transports/remotefile.py +++ b/messagefoundry/transports/remotefile.py @@ -507,15 +507,25 @@ def _make_client( raise ValueError(f"REMOTEFILE protocol must be one of {_PROTOCOLS}, got {protocol!r}") -def _anon_ftp_guard(s: dict[str, Any]) -> InsecureHopGuard | None: +def _anon_ftp_guard( + s: dict[str, Any], + *, + cleartext_accepted: bool = False, + cleartext_reason: str | None = None, + connection: str | None = None, +) -> InsecureHopGuard | None: """An :class:`~messagefoundry.transports.mllp.InsecureHopGuard` for an ANONYMOUS plain-``ftp`` hop (protocol ``ftp`` with no credentials), or ``None`` for any other protocol / a credentialed ftp. Credentialed plain-ftp is already refused by :func:`_validate_common` (it puts the credential itself on the wire in the clear); ``ftps``/``sftp`` are encrypted. The remaining gap #200 closes is an ANONYMOUS plain-ftp hop — no credential, but the message BODY is still PHI over a cleartext channel. - Keyed on the posture gradient off-loopback (refuse production-PHI, warn non-prod PHI, allow - synthetic / loopback / per-connection-attested).""" + Keyed on the shared gradient off-loopback: loopback / per-connection-attested ALLOW, an ADR 0153 + ``cleartext_accepted`` declaration WARNs (loudly, audited), everything else REFUSES under ENFORCE. + + The acceptance pair arrives as arguments rather than out of ``s``: it is a top-level OUTBOUND key, + not a transport setting, and it is **Destination-only** (ADR 0153 decision 2), so the inbound + ``RemoteFileSource`` path leaves it at its default.""" if str(s.get("protocol", "sftp")).lower() != "ftp": return None if s.get("username") or s.get("password"): @@ -528,12 +538,26 @@ def _anon_ftp_guard(s: dict[str, Any]) -> InsecureHopGuard | None: description="cleartext anonymous FTP egress", attested=bool(s.get("tls_hop_attested", False)), attested_reason=None if reason is None else str(reason), + cleartext_accepted=cleartext_accepted, + cleartext_reason=cleartext_reason, + connection=connection, ) -def _validate_common(s: dict[str, Any]) -> str: +def _validate_common( + s: dict[str, Any], + *, + cleartext_accepted: bool = False, + cleartext_reason: str | None = None, + connection: str | None = None, +) -> str: """Shared construction-time validation: required ``host``/``remote_dir``, a known ``protocol``, and - the cleartext-FTP credential guard. Returns the normalized protocol.""" + the cleartext-FTP credential guard. Returns the normalized protocol. + + The ADR 0153 acceptance pair is threaded through to the anonymous-ftp hop guard below — it must + reach the ENFORCED gate that runs here, not just the destination's send-time backstop, or a declared + acceptance would be refused at construction and never take effect. Defaults off, so the inbound + ``RemoteFileSource`` path (which has no such field — Destination-only) is byte-identical.""" for req in ("host", "remote_dir"): if not s.get(req): raise ValueError(f"REMOTEFILE connector requires a {req!r} setting") @@ -560,7 +584,12 @@ def _validate_common(s: dict[str, Any]) -> str: # cleartext. Refuse a production-PHI hop off-loopback at the ENFORCED construction gate (the # credentialed case above is the orthogonal credential-on-the-wire guard). No-op for ftps/sftp/ # credentialed-ftp, and byte-identical off the enforced gate (posture unstamped). - guard = _anon_ftp_guard(s) + guard = _anon_ftp_guard( + s, + cleartext_accepted=cleartext_accepted, + cleartext_reason=cleartext_reason, + connection=connection, + ) if guard is not None: guard.enforce_construction() return protocol @@ -571,10 +600,20 @@ class RemoteFileDestination(DestinationConnector): def __init__(self, config: Destination) -> None: s = config.settings - _validate_common(s) + _validate_common( + s, + cleartext_accepted=config.cleartext_accepted, + cleartext_reason=config.cleartext_reason, + connection=config.name, + ) # #200 send-time backstop for an anonymous plain-ftp hop (the enforced refusal already fired in # _validate_common at the construction gate). None for ftps/sftp/credentialed-ftp. - self._hop_guard = _anon_ftp_guard(s) + self._hop_guard = _anon_ftp_guard( + s, + cleartext_accepted=config.cleartext_accepted, + cleartext_reason=config.cleartext_reason, + connection=config.name, + ) # Constructing the SFTP client validates the host-key escape posture fail-fast (build_check). # #190 (ADR 0093): pass the instance [tls] internal-CA trust-anchor policy so an FTPS hop that # names no tls_ca_file of its own verifies against the org internal CA. diff --git a/messagefoundry/transports/rest.py b/messagefoundry/transports/rest.py index 53fd9cc6..272a94a7 100644 --- a/messagefoundry/transports/rest.py +++ b/messagefoundry/transports/rest.py @@ -47,6 +47,7 @@ HopPosture, InsecureHopRefused, RevocationHopGuard, + cleartext_acceptance_audit_sink, current_hop_posture, enforce_insecure_hop, insecure_hop_disposition, @@ -294,26 +295,55 @@ def _current_hop_posture_fail_closed() -> HopPosture: def _shipped_strict_disposition( - posture: HopPosture, *, host: str, attested: bool + posture: HopPosture, + *, + host: str, + attested: bool, + cleartext_accepted: bool = False, + weakened_tls: bool = False, ) -> HopDisposition: - """The floored posture-keyed disposition for an ALREADY-SHIPPED insecure-egress cell (#200). - - Runs the instance ``posture`` through the ONE authority (:func:`insecure_hop_disposition`) with the - global escape CLAMPED to non-production (:func:`hop_insecure_escape_downgrades`), then applies - decision 5's no-loosen floor: a cell that refused BOTH staging and production PHI today must keep - REFUSE for a non-prod PHI hop that reaches the gradient's WARN *without* the escape (arm 6). Only the - non-prod escape (arm 4 → WARN), a per-hop attestation, on-box loopback, or a synthetic instance - (all → ALLOW) relaxes it. Pure — no I/O — so the send-time guard can reuse it verbatim.""" - audited_opt_out = hop_insecure_escape_downgrades(enforcing=posture.enforcing) + """The floored disposition for an ALREADY-SHIPPED insecure-egress cell (#200, amended by ADR 0153). + + Runs the instance ``posture`` through the ONE authority (:func:`insecure_hop_disposition`), then + applies ADR 0092 decision 5's no-loosen floor: a cell that refused BOTH staging and production PHI + today must keep REFUSE for a hop that only reaches the gradient's WARN because the instance is not + enforcing. Pure apart from one ``os.environ`` read on the ``weakened_tls`` arm, so the send-time + guard can reuse it verbatim. + + **Two different cells share this function, and ADR 0153 governs only one of them.** + + * A **cleartext** hop (``http://`` body or credential egress) is what 0153 decision 2 is about, so + it is keyed on ``cleartext_accepted`` and the blunt ``MEFOR_ALLOW_INSECURE_TLS`` escape no longer + reaches it (decision 5 — the variable "can no longer influence a cleartext-hop decision"). The + floor is re-keyed to match: a WARN reached by the non-enforcing dial alone is still floored to + REFUSE, exactly as before, but a declared acceptance survives it. Without that re-key decision 2 + would be inert for REST/SOAP/FHIR/DICOMweb — the largest cleartext-egress family in the product, + and the only one where the declaration is a genuine escape (``Tcp()``/``X12()`` never reach here). + * A **``verify_tls=false``** hop is NOT cleartext — it is encrypted but unauthenticated — so it is + explicitly OUT of 0153's scope ("the cleartext transport-hop decision only"). It keeps the + pre-0153 clamped global escape (``weakened_tls=True``), which is also what the MLLP/FTPS + ``tls_verify=false`` cells do via ``weakened_tls_escape_permitted_here()``; ``cleartext_accepted`` + deliberately does NOT reach it. Threading the acceptance here would have LOOSENED an enforcing hop + that refuses today (0092 decision 5 forbids that), attached an operator's "this hop has no TLS" + reason to a hop that has TLS, and split the HTTP family from MLLP on the same question. + + The only 0153 change that reaches the verify-off cell is the deleted ``not is_phi`` ALLOW arm, which + can only ever tighten it.""" + # Clamped to non-enforcing upstream, so under ENFORCE this is always False and the blunt escape can + # never satisfy an enforcing hop (ADR 0092 decision 2). Read only for the verify-off cell. + escape = weakened_tls and hop_insecure_escape_downgrades(enforcing=posture.enforcing) + permitted = cleartext_accepted or escape disposition = insecure_hop_disposition( - is_phi=posture.is_phi, enforcing=posture.enforcing, is_loopback_hop=is_loopback_hop_host(host), hop_attested=attested, - audited_opt_out=audited_opt_out, + # Arm 3 of the new precedence occupies exactly the slot the pre-0153 `audited_opt_out` arm did + # (WARN, ahead of the enforcing REFUSE), so feeding the clamped escape through it is + # byte-identical for the verify-off cell. + cleartext_accepted=permitted, ) - if disposition is HopDisposition.WARN and not audited_opt_out: - # arm-6 WARN (non-prod PHI, no escape) — this shipped cell REFUSED it; keep it strict. + if disposition is HopDisposition.WARN and not permitted: + # A WARN reached via the non-enforcing dial alone — this shipped cell REFUSED it; keep it strict. return HopDisposition.REFUSE return disposition @@ -333,11 +363,24 @@ class InsecureHopGuard: posture: HopPosture attested: bool cell: str + # ADR 0153 decision 2 — captured with the posture so the send-time re-assertion sees exactly the + # declaration the construction gate decided on. + cleartext_accepted: bool = False + # True only for the verify_tls=false cell, which is OUT of ADR 0153's scope and keeps the clamped + # global escape (see _shipped_strict_disposition). Captured rather than inferred so the send-time + # re-assertion cannot silently decide a different question than construction did. + weakened_tls: bool = False def assert_send(self, host: str, redacted_url: str) -> None: """Re-assert (zero I/O) that ``host`` is still a permitted hop under the captured posture.""" if ( - _shipped_strict_disposition(self.posture, host=host, attested=self.attested) + _shipped_strict_disposition( + self.posture, + host=host, + attested=self.attested, + cleartext_accepted=self.cleartext_accepted, + weakened_tls=self.weakened_tls, + ) is HopDisposition.REFUSE ): raise InsecureHopRefused( @@ -347,42 +390,111 @@ def assert_send(self, host: str, redacted_url: str) -> None: def _enforce_shipped_hop( - host: str, *, cell: str, message: str, attested: bool + host: str, + *, + cell: str, + message: str, + attested: bool, + cleartext_accepted: bool = False, + cleartext_reason: str | None = None, + connection: str | None = None, + weakened_tls: bool = False, ) -> tuple[HopDisposition, HopPosture]: """Decide + enforce an already-shipped insecure hop at CONSTRUCTION, returning (disposition, posture). Keys on the active (fail-closed) posture, applies the decision-5 floor, then acts via :func:`enforce_insecure_hop` (raise on REFUSE, loud-log on WARN, no-op on ALLOW). When a per-hop - attestation SUPPRESSES a would-be production-PHI refusal it is recorded loudly — the audited opt-in - that replaces the blunt global escape for the production case (decision 3).""" + attestation SUPPRESSES a would-be refusal it is recorded loudly — the audited opt-in that replaced + the blunt global escape (ADR 0092 decision 3). When an ADR 0153 ``cleartext_accepted`` declaration is + what produced the WARN, that is recorded too, at every construction, naming ``connection`` so the + record leads back to the declaration that produced it.""" posture = _current_hop_posture_fail_closed() - disposition = _shipped_strict_disposition(posture, host=host, attested=attested) + disposition = _shipped_strict_disposition( + posture, + host=host, + attested=attested, + cleartext_accepted=cleartext_accepted, + weakened_tls=weakened_tls, + ) + # The `posture.is_phi` conjunct this branch used to carry went with ADR 0153: the authority no longer + # reads the label, so gating the audit on it would silence the record for exactly the hops that + # newly depend on the attestation to cross. if ( disposition is HopDisposition.ALLOW and attested - and posture.is_phi and posture.enforcing and not is_loopback_hop_host(host) ): logger.warning( - "insecure transport hop ATTESTED secure (suppresses a production-PHI refusal) — %s: %s", + "insecure transport hop ATTESTED secure (suppresses an enforcing refusal) — %s: %s", cell, message, ) - enforce_insecure_hop(disposition, message=message, cell=cell) + enforce_insecure_hop( + disposition, + message=message, + cell=cell, + audit_sink=( + cleartext_acceptance_audit_sink(cleartext_reason, connection=connection) + if disposition is HopDisposition.WARN and cleartext_accepted + else None + ), + ) return disposition, posture +def cleartext_acceptance_from_settings( + s: Mapping[str, Any], +) -> tuple[bool, str | None, str | None]: + """``(cleartext_accepted, cleartext_reason, connection)`` off an ``env()``-resolved settings mapping. + + ADR 0153's pair is a **top-level outbound key**, but the deep settings-driven seams — the forward-proxy + credential chain, the HTTP Digest / OAuth2 / SMART token-endpoint providers, and the ``FhirLookup`` + read executor — receive only a settings mapping, exactly as they already do for ``tls_hop_attested``. + The runner's ``_dest_config`` mirrors the declaration into those resolved settings (and only when it + is set, so an outbound that declared nothing is byte-identical), and this is the single reader, so + those resolved keys are never re-parsed by hand at five call sites. + + The connection NAME is mirrored alongside them (``cleartext_connection``) so the acceptance record + from one of these deep seams can still name the declaration that produced it — a record an auditor + cannot trace back to a connection is not much of a record. A ``FhirLookup`` connection, which has no + ``Destination``, carries the pair as a spec setting written by its factory and supplies its own name + directly.""" + reason = s.get("cleartext_reason") + connection = s.get("cleartext_connection") + return ( + bool(s.get("cleartext_accepted", False)), + None if reason is None else str(reason), + None if connection is None else str(connection), + ) + + def refuse_cleartext_credential_hop( - scheme: str, url: str, *, credential: str, attested: bool = False + scheme: str, + url: str, + *, + credential: str, + attested: bool = False, + cleartext_accepted: bool = False, + cleartext_reason: str | None = None, + connection: str | None = None, ) -> None: - """Refuse a named ``credential`` riding a cleartext (``http``) hop (posture-keyed, #200). + """Refuse a named ``credential`` riding a cleartext (``http``) hop (#200, amended by ADR 0153). The header-agnostic core of :func:`refuse_cleartext_credentials`, reused for a credential that does - NOT ride the ``Authorization`` header (a SOAP WS-Security UsernameToken in the body). Re-keyed onto - the ONE authority: a production-PHI hop is REFUSED (the clamped global escape can no longer silence - it — decision 2), a non-prod PHI hop is refused unless the escape downgrades it to a loud WARN, and an - on-box loopback / per-hop-attested / synthetic hop is allowed.""" + NOT ride the ``Authorization`` header (a SOAP WS-Security UsernameToken in the body). Keyed on the + ONE authority: an enforcing hop is REFUSED, an on-box loopback / per-hop-attested hop is allowed, and + an ADR 0153 ``cleartext_accepted`` declaration crosses it with a loud, audited WARN. + + **``cleartext_accepted`` deliberately reaches the credential hops too**, even though putting a + password on the wire is a worse claim than putting a body on it. ADR 0153 is silent here, and the + alternative — leaving credentials with no expressible declaration — would push an operator whose + legacy peer needs Basic auth over a cleartext segment into writing a FALSE ``tls_hop_attested``, + which is precisely the defect 0153 exists to remove. The declaration stays per-connection, WARNs at + every construction, is audited, and is reported as a loosening, so it is strictly more visible than + the blanket escape that used to permit exactly this. (SMTP AUTH over cleartext remains refused + OUTRIGHT in ``transports.email`` — that is a hard refusal, not a posture decision, and is + untouched.)""" if scheme != "http": return host = urllib.parse.urlsplit(url).hostname or "" @@ -391,11 +503,21 @@ def refuse_cleartext_credential_hop( cell="HTTP cleartext credentials", message=f"sends a {credential} over cleartext http to {host!r}", attested=attested, + cleartext_accepted=cleartext_accepted, + cleartext_reason=cleartext_reason, + connection=connection, ) def refuse_cleartext_credentials( - scheme: str, headers: dict[str, str], url: str, *, attested: bool = False + scheme: str, + headers: dict[str, str], + url: str, + *, + attested: bool = False, + cleartext_accepted: bool = False, + cleartext_reason: str | None = None, + connection: str | None = None, ) -> None: """Refuse to send credentials over a cleartext (``http``) channel (posture-keyed, #200). @@ -405,26 +527,37 @@ def refuse_cleartext_credentials( if "Authorization" not in headers: return refuse_cleartext_credential_hop( - scheme, url, credential="credential (Authorization header)", attested=attested + scheme, + url, + credential="credential (Authorization header)", + attested=attested, + cleartext_accepted=cleartext_accepted, + cleartext_reason=cleartext_reason, + connection=connection, ) def refuse_cleartext_egress( - scheme: str, url: str, *, attested: bool = False + scheme: str, + url: str, + *, + attested: bool = False, + cleartext_accepted: bool = False, + cleartext_reason: str | None = None, + connection: str | None = None, ) -> InsecureHopGuard | None: - """Refuse a cleartext (``http``) outbound to a **non-loopback** host (ASVS 12.2.1, posture-keyed #200). + """Refuse a cleartext (``http``) outbound to a **non-loopback** host (ASVS 12.2.1, #200 / ADR 0153). A plaintext ``http://`` destination puts the PHI-bearing request body on the wire even with no - ``Authorization`` header, so an off-box http egress is decided by the instance posture: a - production-PHI hop REFUSES (escape inert — decision 2), a non-prod PHI hop refuses unless the clamped - escape downgrades it to a loud WARN (decision 5 floor keeps it strict otherwise), and an on-box - loopback / per-hop-attested / synthetic hop is allowed — so the default ``127.0.0.1`` posture stays - byte-identical. Complements :func:`refuse_cleartext_credentials`, which fires first (more specific) + ``Authorization`` header, so an off-box http egress is decided by the ONE authority: an enforcing hop + REFUSES, an on-box loopback / per-hop-attested hop is allowed — so the default ``127.0.0.1`` posture + stays byte-identical — and an ADR 0153 ``cleartext_accepted`` declaration crosses it with a loud, + audited WARN. Complements :func:`refuse_cleartext_credentials`, which fires first (more specific) when the connection also carries credentials. - Returns an :class:`InsecureHopGuard` when the cleartext hop was PERMITTED (a warned / attested off-box - egress) so the caller re-asserts it at send; ``None`` for a secure or loopback hop (no send guard - needed — the send stays byte-identical).""" + Returns an :class:`InsecureHopGuard` when the cleartext hop was PERMITTED (a warned / attested / + accepted off-box egress) so the caller re-asserts it at send; ``None`` for a secure or loopback hop + (no send guard needed — the send stays byte-identical).""" if scheme != "http": return None host = urllib.parse.urlsplit(url).hostname or "" @@ -433,10 +566,18 @@ def refuse_cleartext_egress( cell="HTTP cleartext egress", message=(f"delivers its payload over cleartext http to a non-loopback host ({host!r})"), attested=attested, + cleartext_accepted=cleartext_accepted, + cleartext_reason=cleartext_reason, + connection=connection, ) if is_loopback_hop_host(host): return None # on-box loopback — not a network exposure, so no send-time guard - return InsecureHopGuard(posture=posture, attested=attested, cell="HTTP cleartext egress") + return InsecureHopGuard( + posture=posture, + attested=attested, + cell="HTTP cleartext egress", + cleartext_accepted=cleartext_accepted, + ) def refuse_verify_off( @@ -444,12 +585,20 @@ def refuse_verify_off( ) -> InsecureHopGuard | None: """Refuse a ``verify_tls=false`` (unverified-TLS) hop to a non-loopback host (posture-keyed, #200). - Disabling certificate verification makes the ``https`` hop MITM-able, so it is an insecure hop and is - decided exactly like cleartext egress: production-PHI REFUSES (escape inert), a non-prod PHI hop - refuses unless the clamped escape downgrades it to a loud WARN, an on-box loopback / attested / - synthetic hop is allowed. Only meaningful for ``https`` (an ``http`` url has no TLS to verify and is - handled by :func:`refuse_cleartext_egress`); returns ``None`` for a non-https scheme. Returns an - :class:`InsecureHopGuard` when the hop was permitted (a warned / attested off-box hop).""" + Disabling certificate verification makes the ``https`` hop MITM-able, so it is an insecure hop: + an enforcing hop REFUSES, an on-box loopback / per-hop-attested hop is allowed, and the CLAMPED + global escape (``MEFOR_ALLOW_INSECURE_TLS``, inert under ENFORCE) downgrades a non-enforcing one to + a loud WARN. Only meaningful for ``https`` (an ``http`` url has no TLS to verify and is handled by + :func:`refuse_cleartext_egress`); returns ``None`` for a non-https scheme. Returns an + :class:`InsecureHopGuard` when the hop was permitted (a warned / attested off-box hop). + + **ADR 0153's ``cleartext_accepted`` deliberately does NOT reach this cell.** A verify-off hop is + encrypted-but-unauthenticated, not cleartext, and 0153 scopes itself to "the cleartext + transport-hop decision only". Threading the acceptance here would LOOSEN an enforcing hop that + refuses today (ADR 0092 decision 5 forbids that), attach an operator's "this peer cannot do TLS" + reason to a peer that plainly does, and split this cell from the MLLP/FTPS ``tls_verify=false`` + cells, which decide the same question through ``weakened_tls_escape_permitted_here()``. The one + 0153 change that does reach here is the deleted ``not is_phi`` ALLOW arm — a tightening.""" if scheme != "https": return None host = urllib.parse.urlsplit(url).hostname or "" @@ -459,10 +608,11 @@ def refuse_verify_off( cell=cell, message=f"disables TLS certificate verification for non-loopback host {host!r}", attested=attested, + weakened_tls=True, ) if is_loopback_hop_host(host): return None - return InsecureHopGuard(posture=posture, attested=attested, cell=cell) + return InsecureHopGuard(posture=posture, attested=attested, cell=cell, weakened_tls=True) def refuse_unrevoked_verified_hop( @@ -636,6 +786,9 @@ def proxy_auth_handler_from_settings( proxy_scheme: str, dest_scheme: str, attested: bool, + cleartext_accepted: bool = False, + cleartext_reason: str | None = None, + connection: str | None = None, ) -> tuple[tuple[tuple[str, str], ...], _ProxyDigestRecipe | None]: """The #127 proxy-credential-type dispatch: returns ``(pre-emptive auth header, reactive digest recipe)`` for an already-``env()``-resolved settings mapping ``s``. (Named per the phase doc; lives @@ -665,8 +818,17 @@ def proxy_auth_handler_from_settings( ) # A proxy credential over a cleartext http proxy hop crosses in the clear on the CONNECT/request — # refuse it REGARDLESS of destination scheme, posture-keyed (ADR 0092). Names only the proxy host. + # ADR 0153: the proxy hop is a distinct crossing from the destination hop, but it belongs to the + # SAME connection, so it is governed by that connection's one declaration. Threaded explicitly (like + # `attested` beside it) rather than read off the settings mapping — one declaration, one source. refuse_cleartext_credential_hop( - proxy_scheme, proxy_url, credential="web-proxy credential", attested=attested + proxy_scheme, + proxy_url, + credential="web-proxy credential", + attested=attested, + cleartext_accepted=cleartext_accepted, + cleartext_reason=cleartext_reason, + connection=connection, ) kind = str(auth_type or "basic").strip().lower() if kind == "basic": @@ -693,7 +855,13 @@ def proxy_auth_handler_from_settings( def proxy_config_from_settings( - s: Mapping[str, Any], *, dest_scheme: str, attested: bool = False + s: Mapping[str, Any], + *, + dest_scheme: str, + attested: bool = False, + cleartext_accepted: bool = False, + cleartext_reason: str | None = None, + connection: str | None = None, ) -> ProxyConfig | None: """Build the per-connection :class:`ProxyConfig` from an already-``env()``-resolved settings mapping, or ``None`` when no proxy is configured (byte-identical). Reads ``proxy_url`` (#112), ``proxy_no_proxy`` @@ -735,6 +903,9 @@ def proxy_config_from_settings( proxy_scheme=proxy_scheme, dest_scheme=dest_scheme, attested=attested, + cleartext_accepted=cleartext_accepted, + cleartext_reason=cleartext_reason, + connection=connection, ) return ProxyConfig( proxies=(("http", proxy_url), ("https", proxy_url)), @@ -799,7 +970,13 @@ def ech_sidecar_url_from_settings(s: Mapping[str, Any]) -> str | None: def egress_route_from_settings( - s: Mapping[str, Any], *, dest_scheme: str, attested: bool = False + s: Mapping[str, Any], + *, + dest_scheme: str, + attested: bool = False, + cleartext_accepted: bool = False, + cleartext_reason: str | None = None, + connection: str | None = None, ) -> ProxyConfig | None: """Resolve the per-connection forward/egress **proxy** (ADR 0126) from ``proxy_url``, or ``None`` (byte-identical). **Fails closed on ``ech_egress``:** the ECH SNI-hiding send-path (ADR 0139) is @@ -813,7 +990,14 @@ def egress_route_from_settings( "build; on this connector it would NOT hide the SNI, so it is refused rather than silently " "leaking it (ADR 0139)" ) - return proxy_config_from_settings(s, dest_scheme=dest_scheme, attested=attested) + return proxy_config_from_settings( + s, + dest_scheme=dest_scheme, + attested=attested, + cleartext_accepted=cleartext_accepted, + cleartext_reason=cleartext_reason, + connection=connection, + ) class RestDestination(DestinationConnector): @@ -846,6 +1030,9 @@ def __init__(self, config: Destination) -> None: self.consumes_metadata: bool = bool(s.get("dynamic_headers", False)) # #200 (ADR 0092): the per-connection insecure-hop attestation, keying the posture-keyed refusal. attested = config.tls_hop_attested + # ADR 0153 decision 2: the OPPOSITE per-connection declaration — this hop is cleartext, is not + # secure, and that is accepted (WARN + audit, never a silent ALLOW). + accepted, accept_reason = config.cleartext_accepted, config.cleartext_reason # Captured at construction; re-asserted (zero I/O) at the byte-crossing in _post (decision 4). self._hop_guard: InsecureHopGuard | None = None # ADR 0139 (ASVS 12.1.5): opt-in ECH SNI-hiding. When set, this connection re-addresses each @@ -862,7 +1049,14 @@ def __init__(self, config: Destination) -> None: self._proxy: ProxyConfig | None = ( None if self._ech_sidecar is not None - else egress_route_from_settings(s, dest_scheme=scheme, attested=attested) + else egress_route_from_settings( + s, + dest_scheme=scheme, + attested=attested, + cleartext_accepted=accepted, + cleartext_reason=accept_reason, + connection=config.name, + ) ) dest_host = urllib.parse.urlsplit(self.url).hostname or "" proxy_dest = self._proxy.for_host(dest_host) if self._proxy is not None else None @@ -873,10 +1067,25 @@ def __init__(self, config: Destination) -> None: # CONNECT tunnel for an https destination, sends it to the proxy for an http one (ADR 0126). self._headers.update(proxy_dest.auth_headers()) enforce_outbound_length_limits(self.url, self._headers) - refuse_cleartext_credentials(scheme, self._headers, self.url, attested=attested) + refuse_cleartext_credentials( + scheme, + self._headers, + self.url, + attested=attested, + cleartext_accepted=accepted, + cleartext_reason=accept_reason, + connection=config.name, + ) # ASVS 12.2.1: even without an Authorization header the request body is PHI, so a cleartext # http egress to a non-loopback host is refused (loopback stays byte-identical). See rest.py. - self._hop_guard = refuse_cleartext_egress(scheme, self.url, attested=attested) + self._hop_guard = refuse_cleartext_egress( + scheme, + self.url, + attested=attested, + cleartext_accepted=accepted, + cleartext_reason=accept_reason, + connection=config.name, + ) # ASVS 4.1.5 (ADR 0018): opt-in detached-JWS signing of the outbound body. None = off (byte- # identical). Built here so a bad key/algorithm fails loud at connector construction (check/ # dry-run/start), like a bad TLS cert; the per-request signature is minted in _post (off-loop). @@ -896,7 +1105,13 @@ def __init__(self, config: Destination) -> None: # SMART access token never ships over cleartext http (the detached-JWS signature, by # contrast, is public-verifiable and needs no such guard). refuse_cleartext_credentials( - scheme, {**self._headers, "Authorization": "Bearer"}, self.url, attested=attested + scheme, + {**self._headers, "Authorization": "Bearer"}, + self.url, + attested=attested, + cleartext_accepted=accepted, + cleartext_reason=accept_reason, + connection=config.name, ) if bool(s.get("verify_tls", True)): # #201 (ADR 0078 amendment): the verify-ON https hop validates the peer cert but does no @@ -925,10 +1140,14 @@ def __init__(self, config: Destination) -> None: self._opener = _NO_REDIRECT_OPENER else: # verify_tls=false makes the https hop MITM-able — an insecure hop decided by the instance - # posture (#200): production-PHI REFUSES (escape inert), a non-prod PHI hop refuses unless the - # clamped escape / a per-hop attestation permits it. Loopback stays byte-identical. + # posture (#200): an enforcing hop REFUSES, a non-enforcing one refuses unless the clamped + # escape / a per-hop attestation permits it. Loopback stays byte-identical. ADR 0153's + # cleartext_accepted does NOT reach here — this hop has TLS (see refuse_verify_off). guard = refuse_verify_off( - scheme, self.url, connector="REST destination", attested=attested + scheme, + self.url, + connector="REST destination", + attested=attested, ) if guard is not None: self._hop_guard = guard diff --git a/messagefoundry/transports/smart.py b/messagefoundry/transports/smart.py index b7847a9c..e982b1a6 100644 --- a/messagefoundry/transports/smart.py +++ b/messagefoundry/transports/smart.py @@ -45,7 +45,7 @@ from typing import TYPE_CHECKING, Any from messagefoundry.config.models import ConnectorType, Destination, SignatureAlgorithm -from messagefoundry.config.settings import INSECURE_TLS_ESCAPE_ENV, insecure_tls_allowed +from messagefoundry.config.tls_policy import InsecureHopRefused from messagefoundry.transports.base import DeliveryError # Reuse rest.py's hardened opener + URL redaction (no new HTTP plumbing) — exactly as fhir.py/soap.py @@ -55,6 +55,8 @@ ProxyConfig, _no_redirect_opener, _redact_url, + cleartext_acceptance_from_settings, + refuse_cleartext_credential_hop, ) from messagefoundry.transports.signing import CompactJwtSigner @@ -109,6 +111,10 @@ def __init__( private_key_password: str | None = None, expiry_skew_seconds: float = _DEFAULT_EXPIRY_SKEW, timeout_seconds: float = _DEFAULT_TOKEN_TIMEOUT, + attested: bool = False, + cleartext_accepted: bool = False, + cleartext_reason: str | None = None, + connection: str | None = None, proxy: ProxyConfig | None = None, ) -> None: if not token_url: @@ -116,12 +122,30 @@ def __init__( scheme = urllib.parse.urlsplit(token_url).scheme.lower() if scheme not in ("http", "https"): raise SmartAuthError(f"smart_token_url must be http or https, got scheme {scheme!r}") - if scheme == "http" and not insecure_tls_allowed(): - # The client_assertion JWT is a credential — refuse to send it over cleartext. - raise SmartAuthError( - "SMART token endpoint over cleartext http would expose the client_assertion; " - f"refused unless {INSECURE_TLS_ESCAPE_ENV} is set (dev/trusted-network only) — use https" + # The client_assertion JWT is a credential, so this hop goes through the ONE posture-keyed + # authority — exactly like its OAuth2 sibling in http_auth.py and the delivery cells. It used to + # read the raw, UNCLAMPED `MEFOR_ALLOW_INSECURE_TLS`, which meant one process-wide environment + # variable put a signed client_assertion on cleartext http even on an enforcing PHI instance, + # and left the variable alive here while ADR 0153 decision 5 unhooked it everywhere else. + # `refuse_cleartext_credential_hop` raises `InsecureHopRefused` (a `tls_policy` `ValueError`) on + # REFUSE; re-raise as `SmartAuthError` to preserve THIS seam's error contract (both are + # `ValueError` subclasses, so the loader surfaces either identically). Never echoes the key. + try: + refuse_cleartext_credential_hop( + scheme, + token_url, + credential="SMART client_assertion", + attested=attested, + cleartext_accepted=cleartext_accepted, + cleartext_reason=cleartext_reason, + connection=connection, ) + except InsecureHopRefused as exc: + raise SmartAuthError( + "SMART token endpoint over cleartext http would expose the client_assertion; refused " + "by the instance security posture (use https, attest the hop as secure via " + "tls_hop_attested, or declare cleartext_accepted with a cleartext_reason)" + ) from exc if not client_id: raise SmartAuthError("SMART Backend Services requires a 'smart_client_id' setting") if not private_key: @@ -263,6 +287,10 @@ def token_provider_from_settings( return None if not s.get("smart_enabled", True): return None + # ADR 0153: the same per-connection declaration the delivery hop carries, mirrored into these + # resolved settings by the runner's _dest_config (with the connection name, so the acceptance audit + # record names the declaration). Read exactly as the OAuth2 sibling does. + accepted = cleartext_acceptance_from_settings(s) return SmartBackendTokenProvider( token_url=str(s.get("smart_token_url") or ""), client_id=str(s.get("smart_client_id") or ""), @@ -276,6 +304,12 @@ def token_provider_from_settings( ), expiry_skew_seconds=float(s.get("smart_expiry_skew_seconds", _DEFAULT_EXPIRY_SKEW)), timeout_seconds=float(s.get("smart_timeout_seconds", _DEFAULT_TOKEN_TIMEOUT)), + # #200: the per-connection insecure-hop attestation keys the posture-keyed cleartext refusal in + # __init__ (read from settings exactly as _dest_config / the OAuth2 provider do). + attested=bool(s.get("tls_hop_attested", False)), + cleartext_accepted=accepted[0], + cleartext_reason=accepted[1], + connection=accepted[2], proxy=proxy, # ADR 0126: forward-proxy the token-endpoint POST ) diff --git a/messagefoundry/transports/soap.py b/messagefoundry/transports/soap.py index 45137388..59473ca5 100644 --- a/messagefoundry/transports/soap.py +++ b/messagefoundry/transports/soap.py @@ -331,13 +331,26 @@ def __init__(self, config: Destination) -> None: # #200 (ADR 0092): the per-connection insecure-hop attestation, keying the posture-keyed refusal. attested = config.tls_hop_attested + # ADR 0153 decision 2: the OPPOSITE per-connection declaration — this hop is cleartext, is not + # secure, and that is accepted (WARN + audit, never a silent ALLOW). Held on the instance because + # the WS-Security / body-secret credential hops below are decided in helper methods. + self._cleartext_accepted = config.cleartext_accepted + self._cleartext_reason = config.cleartext_reason + # Held for the same reason: the acceptance audit record names the declaring connection. + self._connection_name = config.name + accepted, accept_reason = self._cleartext_accepted, self._cleartext_reason # Captured at construction; re-asserted (zero I/O) at the byte-crossing in _post (decision 4). self._hop_guard: InsecureHopGuard | None = None self._validate_ws(scheme, s, attested=attested) # BACKLOG #112/#127/#128 (ADR 0126): per-connection forward/egress proxy (None → byte-identical). self._proxy: ProxyConfig | None = egress_route_from_settings( - s, dest_scheme=scheme, attested=attested + s, + dest_scheme=scheme, + attested=attested, + cleartext_accepted=accepted, + cleartext_reason=accept_reason, + connection=config.name, ) dest_host = urllib.parse.urlsplit(self.url).hostname or "" proxy_dest = self._proxy.for_host(dest_host) if self._proxy is not None else None @@ -348,10 +361,25 @@ def __init__(self, config: Destination) -> None: # Pre-emptive Proxy-Authorization (Basic; empty for Digest/none) — tunnelled for https (0126). self._headers.update(proxy_dest.auth_headers()) enforce_outbound_length_limits(self.url, self._headers) - refuse_cleartext_credentials(scheme, self._headers, self.url, attested=attested) + refuse_cleartext_credentials( + scheme, + self._headers, + self.url, + attested=attested, + cleartext_accepted=accepted, + cleartext_reason=accept_reason, + connection=config.name, + ) # ASVS 12.2.1: the SOAP envelope body is PHI, so a cleartext http egress to a non-loopback # host is refused even without credentials (loopback stays byte-identical). See rest.py. - self._hop_guard = refuse_cleartext_egress(scheme, self.url, attested=attested) + self._hop_guard = refuse_cleartext_egress( + scheme, + self.url, + attested=attested, + cleartext_accepted=accepted, + cleartext_reason=accept_reason, + connection=config.name, + ) # ASVS 4.1.5 (ADR 0018): opt-in detached-JWS signing of the outbound envelope. None = off # (byte-identical). Built here so a bad key/algorithm fails loud at connector construction; the # signature is minted in _post over the FINAL wire bytes (the WS-* wrapped envelope, ADR 0015). @@ -417,7 +445,13 @@ def __init__(self, config: Destination) -> None: self._token_provider = bearer_provider_from_settings(s, proxy=self._proxy) if self._token_provider is not None: refuse_cleartext_credentials( - scheme, {**self._headers, "Authorization": "Bearer"}, self.url, attested=attested + scheme, + {**self._headers, "Authorization": "Bearer"}, + self.url, + attested=attested, + cleartext_accepted=accepted, + cleartext_reason=accept_reason, + connection=config.name, ) digest = digest_handler_from_settings(s, url=self.url) if digest is not None: @@ -487,7 +521,13 @@ def _parse_body_secrets( # like the WS-Security UsernameToken — refuse it under the same posture gate (loopback / attested # stay allowed, byte-identical). refuse_cleartext_credential_hop( - scheme, self.url, credential="SOAP body secret", attested=attested + scheme, + self.url, + credential="SOAP body secret", + attested=attested, + cleartext_accepted=self._cleartext_accepted, + cleartext_reason=self._cleartext_reason, + connection=self._connection_name, ) return tuple(pairs) @@ -560,6 +600,9 @@ def _validate_ws(self, scheme: str, s: dict[str, Any], *, attested: bool = False self.url, credential="WS-Security UsernameToken credential", attested=attested, + cleartext_accepted=self._cleartext_accepted, + cleartext_reason=self._cleartext_reason, + connection=self._connection_name, ) def _build_headers(self, s: dict[str, Any]) -> dict[str, str]: diff --git a/messagefoundry/transports/tcp.py b/messagefoundry/transports/tcp.py index 903aa5ab..d6bf7674 100644 --- a/messagefoundry/transports/tcp.py +++ b/messagefoundry/transports/tcp.py @@ -136,9 +136,10 @@ def __init__(self, config: Destination) -> None: self._closed = False #: Reconnects observed (stale-detect, post-error discard, desync guard) — log-only. self.reconnects: int = 0 - # #200 (ADR 0092): raw TCP has NO TLS option, so every off-loopback egress is a cleartext PHI hop. - # Refuse a production-PHI hop at the enforced construction gate; allow loopback / synthetic / - # per-connection-attested hops (tls_hop_attested for a trusted-segment / proxy-terminated hop). + # #200 (ADR 0092): raw TCP has NO TLS option, so every off-loopback egress is a cleartext hop. + # Refuse it at the enforced construction gate; allow loopback / per-connection-attested hops + # (tls_hop_attested for a trusted-segment / proxy-terminated hop), or cross it with a loud, + # audited WARN on a `cleartext_accepted` declaration. ADR 0153: no data label relaxes this. self._hop_guard = InsecureHopGuard.capture( host=self.host, port=self.port, @@ -146,6 +147,12 @@ def __init__(self, config: Destination) -> None: description="cleartext raw-TCP egress", attested=config.tls_hop_attested, attested_reason=config.tls_hop_attested_reason, + # ADR 0153 decision 4: raw TCP has NO TLS support at all — no `tls` parameter, no ssl import + # — so here `cleartext_accepted` is a PERMANENT, STRUCTURAL declaration, not a transitional + # one. There is no `tls = true` for it to migrate to (BACKLOG #311). + cleartext_accepted=config.cleartext_accepted, + cleartext_reason=config.cleartext_reason, + connection=config.name, ) self._hop_guard.enforce_construction() diff --git a/messagefoundry/transports/x12.py b/messagefoundry/transports/x12.py index c479dd9a..95da09db 100644 --- a/messagefoundry/transports/x12.py +++ b/messagefoundry/transports/x12.py @@ -110,9 +110,10 @@ def __init__(self, config: Destination) -> None: self._closed = False #: Reconnects observed (stale-detect, post-error discard, desync guard) — log-only. self.reconnects: int = 0 - # #200 (ADR 0092): X12-over-TCP has NO TLS at all, so every off-loopback egress is a cleartext PHI - # hop. Refuse a production-PHI hop at the enforced construction gate; allow loopback / synthetic / - # per-connection-attested hops (tls_hop_attested for a trusted-segment / proxy-terminated hop). + # #200 (ADR 0092): X12-over-TCP has NO TLS at all, so every off-loopback egress is a cleartext + # hop. Refuse it at the enforced construction gate; allow loopback / per-connection-attested hops + # (tls_hop_attested for a trusted-segment / proxy-terminated hop), or cross it with a loud, + # audited WARN on a `cleartext_accepted` declaration. ADR 0153: no data label relaxes this. self._hop_guard = InsecureHopGuard.capture( host=self.host, port=self.port, @@ -120,6 +121,11 @@ def __init__(self, config: Destination) -> None: description="cleartext X12-over-TCP egress", attested=config.tls_hop_attested, attested_reason=config.tls_hop_attested_reason, + # ADR 0153 decision 4: X12-over-TCP has NO TLS support at all, so `cleartext_accepted` is a + # PERMANENT, STRUCTURAL declaration here — there is no `tls = true` to migrate to (#311). + cleartext_accepted=config.cleartext_accepted, + cleartext_reason=config.cleartext_reason, + connection=config.name, ) self._hop_guard.enforce_construction() diff --git a/messagefoundry_webconsole/__init__.py b/messagefoundry_webconsole/__init__.py index d69b7d8c..5d9a7203 100644 --- a/messagefoundry_webconsole/__init__.py +++ b/messagefoundry_webconsole/__init__.py @@ -45,7 +45,7 @@ # If cross-seam support is ever genuinely wanted, re-widen this set AND add the CI matrix that # installs the MIN and MAX supported engine builds — the claim and its test land together, or not # at all. -SUPPORTED_ENGINE_SEAMS: frozenset[int] = frozenset({14}) +SUPPORTED_ENGINE_SEAMS: frozenset[int] = frozenset({15}) #: The vendored static assets shipped in THIS wheel (mounted at /ui/static by :func:`mount_ui`). STATIC_DIR = Path(__file__).parent / "static" diff --git a/tests/conftest.py b/tests/conftest.py index bce0e4de..6bb7a79e 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -127,9 +127,14 @@ def _force_aad_bind_when_requested() -> Iterator[None]: ``mfenc:v2`` writer (``write_v2=True``) at its single construction chokepoint. Re-running the store round-trip suites under this flag turns every real write→read path into an AAD round-trip, so a mismatched encrypt/decrypt cell (a half-threaded ``cell_aad``) surfaces as a ``CipherError`` — the - decisive check that no cell was missed. OFF by default (the suite runs the frozen v1 writer, - byte-identical at rest), so the flag is opt-in for the dedicated aad_bind sweep and never perturbs - the default run or the v1-format assertions in ``test_store_encryption.py``.""" + decisive check that no cell was missed. + + The flag is OFF by default and stays meaningful even though ``[store].aad_bind`` now DEFAULTS TRUE + (ADR 0148 GIVEN 1). It is not a duplicate of that default: it patches ``AesGcmCipher.__init__``, so + it forces ``write_v2`` on ciphers built with an EXPLICIT ``write_v2=False`` — including the ones + ``test_store_encryption.py`` constructs directly to pin the v1 format. The settings default governs + what ``open_store`` builds; this flag governs every cipher in the process, which is what makes the + sweep exhaustive rather than merely representative.""" if os.environ.get("MEFOR_TEST_FORCE_AAD_BIND") != "1": yield return diff --git a/tests/golden/webconsole_seam.snapshot b/tests/golden/webconsole_seam.snapshot index ae63c501..8a15bd29 100644 --- a/tests/golden/webconsole_seam.snapshot +++ b/tests/golden/webconsole_seam.snapshot @@ -5,7 +5,7 @@ # This is a GOLDEN gate: any diff means the seam contract changed - see the test's failure hint. ## ENGINE_UI_SEAM -14 +15 ## dataclass messagefoundry.api._ui_seam.UiDeps engine_seam @@ -166,7 +166,7 @@ MetricsHistorySample: outbox_by_status, ts PendingApprovalResponse: approval_id, detail, operation, status ReloadRequest: config_dir, dry_run ReloadResult: dry_run, handlers, inbound, outbound, routers, running -SecurityPosture: allow_unencrypted_phi, backend, client_address_monoculture, client_denied_last, client_network_denials, data_class, encryption_enabled, enforcement, environment, fips_mode, key_id, key_source, loosenings, memory_encryption_note, memory_encryption_operator_declared, memory_encryption_readout_contradicts_declaration, memory_encryption_readout_source, memory_encryption_self_reported_active, memory_encryption_self_reported_capability, memory_encryption_self_reported_mechanism, openssl_version, plaintext_columns, production, require_encryption, security, synthetic_relaxation +SecurityPosture: allow_unencrypted_phi, backend, client_address_monoculture, client_denied_last, client_network_denials, data_class, encryption_enabled, enforcement, environment, fips_mode, key_id, key_source, loosenings, loosenings_scope, memory_encryption_note, memory_encryption_operator_declared, memory_encryption_readout_contradicts_declaration, memory_encryption_readout_source, memory_encryption_self_reported_active, memory_encryption_self_reported_capability, memory_encryption_self_reported_mechanism, openssl_version, plaintext_columns, production, require_encryption, security, synthetic_relaxation ServiceStatusInfo: enabled, service_name, state StatsResetRequest: all, targets StatsResetTarget: channel_id, destination, role diff --git a/tests/test_ad_session_reconcile.py b/tests/test_ad_session_reconcile.py index 81070658..ff1d962b 100644 --- a/tests/test_ad_session_reconcile.py +++ b/tests/test_ad_session_reconcile.py @@ -221,10 +221,13 @@ def test_breaker_ceiling_reports_the_larger_of_the_two_thresholds() -> None: # --- settings -------------------------------------------------------------------- -def test_reconciler_is_off_by_default() -> None: - """The upgrade must be byte-identical: 0 = disabled, and every safety knob has a default.""" +def test_reconciler_is_on_by_default() -> None: + """The shipped default runs the reconciler (ADR 0148 GIVEN 1), and every safety knob has a default. + + 300 s, not 0: the hardened path is the shipped path. It stays INERT without AD — `should_reconcile` + also requires an LDAP client — so a non-AD deployment creates no task and issues no bind.""" defaults = AuthSettings() - assert defaults.ad_session_recheck_seconds == 0 + assert defaults.ad_session_recheck_seconds == 300 assert defaults.ad_session_recheck_strikes == 2 assert defaults.ad_session_revoke_max == 5 assert defaults.ad_session_revoke_max_fraction == pytest.approx(0.34) diff --git a/tests/test_client_network_allowlist.py b/tests/test_client_network_allowlist.py index b2ef1bdf..92d74ca5 100644 --- a/tests/test_client_network_allowlist.py +++ b/tests/test_client_network_allowlist.py @@ -35,12 +35,23 @@ AuthSettings, SecuritySettings, ServiceSettings, + StoreSettings, load_settings, security_loosenings, ) from messagefoundry.netaddr import client_network_allowed, peer_ip_allowed from messagefoundry.pipeline import Engine + +def _loosenings(sec: SecuritySettings) -> list[tuple[str, str]]: + """``security_loosenings`` with the shipped [store]/[auth] defaults and an empty accepted set. + + The registry takes all four inputs as REQUIRED arguments deliberately (ADR 0148: one posture, and a + deviation the registry cannot see is a second posture by the back door). The tests below are about + the ``[security]`` switches specifically, so the other three are pinned at shipped values here.""" + return security_loosenings(sec, StoreSettings(), AuthSettings(), ()) + + PW = "a-strong-test-passphrase" # >=15, no app/vendor terms — satisfies the ASVS policy WARD = ["10.0.0.0/8"] @@ -154,7 +165,7 @@ def test_default_is_empty_and_is_not_a_loosening() -> None: assert s.allowed_client_networks == [] assert s.client_networks == () # An empty list on the default loopback bind is the SECURE position, not a loosening. - assert security_loosenings(s) == [] + assert _loosenings(s) == [] def test_entries_are_normalized_at_load() -> None: @@ -207,14 +218,14 @@ def test_toml_load_and_malformed_toml_refusal(tmp_path: Path) -> None: def test_exposure_loosening_fires_on_exposure_not_on_the_bind() -> None: # R1: an off-box bind with no allow-list. - names = [n for n, _ in security_loosenings(SecuritySettings(local_access_only=False))] + names = [n for n, _ in _loosenings(SecuritySettings(local_access_only=False))] assert "allowed_client_networks" in names # R2 — the REGRESSION GUARD. The recommended off-box topology keeps the loopback bind and puts a # reverse proxy in front, so a `not local_access_only` test alone would never fire in the # most-exposed supported posture. names = [ n - for n, _ in security_loosenings( + for n, _ in _loosenings( SecuritySettings(web_console_public_address="https://ops.example.com") ) ] @@ -226,7 +237,7 @@ def test_exposure_loosening_fires_on_exposure_not_on_the_bind() -> None: web_console_public_address="https://ops.example.com", allowed_client_networks=WARD ), ): - assert "allowed_client_networks" not in [n for n, _ in security_loosenings(sec)] + assert "allowed_client_networks" not in [n for n, _ in _loosenings(sec)] # --- the trusted_proxies pairing (a broad range would nullify the whole control) ------------------- diff --git a/tests/test_connections_cli.py b/tests/test_connections_cli.py index 9df8ff05..1b317ff6 100644 --- a/tests/test_connections_cli.py +++ b/tests/test_connections_cli.py @@ -195,6 +195,12 @@ def test_hand_comment_survives_gui_upsert(cfg: Path, capsys: pytest.CaptureFixtu "name": "OB", "transport": "mllp", "settings": {"host": "epic.example", "port": 2700}, + # ADR 0153: `connection upsert` validates the edit through the posture-stamped build_check, and + # a plaintext MLLP hop to an off-box host now REFUSES with no data-label carve-out. Declaring + # the acceptance is what makes the edit load-legal — and proves the new keys survive the + # comment-preserving writer, which is the property this test is really about. + "cleartext_accepted": True, + "cleartext_reason": "legacy partner has no MLLP-over-TLS listener", } assert _upsert(cfg, obj, capsys, svc=_svc(cfg))[0] == 0 text = (cfg / "connections.toml").read_text(encoding="utf-8") @@ -317,6 +323,8 @@ def test_cli_end_to_end_maximal_with_commented_sibling( transport = "mllp" ordering = "fifo" dead_letter_days = 7 + cleartext_accepted = true + cleartext_reason = "legacy partner has no MLLP-over-TLS listener" [outbound.settings] host = "epic.example" port = 2700 diff --git a/tests/test_connections_roundtrip.py b/tests/test_connections_roundtrip.py index b6ae4a2f..c58fa0b0 100644 --- a/tests/test_connections_roundtrip.py +++ b/tests/test_connections_roundtrip.py @@ -120,6 +120,10 @@ def _noop_validate(_config_dir: Path) -> None: "deployed": False, "flagged": True, # #131 (ADR 0007 amendment): the object-of-interest flag round-trips too. "waiting_display_delay": 2.5, # #136 (ADR 0065 amendment): the cosmetic waiting-for-reply delay. + # ADR 0153: the per-outbound cleartext-hop acceptance. It MUST round-trip — dropping it on a GUI + # save would silently turn a declared, reviewed hop into a refused one at the next reload. + "cleartext_accepted": True, + "cleartext_reason": "legacy partner has no TLS listener", } _IB_MAX: dict[str, Any] = {**_IB_BASE, **_INBOUND_CASES} @@ -129,12 +133,16 @@ def _noop_validate(_config_dir: Path) -> None: # a pruning size threshold does nothing without its window, so the loader rejects it alone. _COMPANIONS: dict[str, dict[str, Any]] = { "prune_documents_min_bytes": {"prune_documents_after": 30}, + # ADR 0153's pair is load-validated TOGETHER (flag without reason, or reason without flag, both fail + # loud), so each half needs the other for its single-key round-trip to be load-legal. + "cleartext_accepted": {"cleartext_reason": "legacy partner has no TLS listener"}, + "cleartext_reason": {"cleartext_accepted": True}, } def test_case_tables_cover_the_read_schema_exactly() -> None: """Completeness pin: the parametrized cases (plus the base's name/transport) must cover the read - schema exactly — 33 distinct keys (25 inbound + 16 outbound, 8 shared; 41 per-direction slots). + schema exactly — 35 distinct keys (25 inbound + 18 outbound, 8 shared; 43 per-direction slots). A key added to _INBOUND_KEYS/_OUTBOUND_KEYS without a round-trip case fails HERE.""" assert set(_INBOUND_CASES) | {"name", "transport"} == _INBOUND_KEYS assert set(_OUTBOUND_CASES) | {"name", "transport"} == _OUTBOUND_KEYS @@ -155,7 +163,9 @@ def test_inbound_key_roundtrips(tmp_path: Path, key: str) -> None: @pytest.mark.parametrize("key", sorted(_OUTBOUND_CASES)) def test_outbound_key_roundtrips(tmp_path: Path, key: str) -> None: cfg = _config(tmp_path) - obj = {**_OB_BASE, key: _OUTBOUND_CASES[key]} + # _COMPANIONS applies on BOTH directions: ADR 0153's cleartext pair is the first OUTBOUND key whose + # read schema requires a companion, so this call site had none to honour until now. + obj = {**_OB_BASE, **_COMPANIONS.get(key, {}), key: _OUTBOUND_CASES[key]} upsert_connection(cfg, obj, validate=_noop_validate) [entry] = list_connections(cfg) assert entry[key] == _OUTBOUND_CASES[key] diff --git a/tests/test_dicomweb.py b/tests/test_dicomweb.py index 106d2226..b421f41a 100644 --- a/tests/test_dicomweb.py +++ b/tests/test_dicomweb.py @@ -168,20 +168,24 @@ def test_dicomweb_cleartext_http_loopback_allowed() -> None: assert isinstance(dest, DicomWebDestination) -def test_dicomweb_cleartext_http_nonloopback_allowed_with_escape( +def test_dicomweb_cleartext_http_nonloopback_allowed_when_accepted( monkeypatch: pytest.MonkeyPatch, ) -> None: - monkeypatch.setenv("MEFOR_ALLOW_INSECURE_TLS", "1") - # #200 (ADR 0092): the escape downgrades REFUSE→WARN only on a NON-production instance (decision 2). - with active_hop_posture(HopPosture(is_phi=True, enforcing=False)): + # ADR 0153: the blunt MEFOR_ALLOW_INSECURE_TLS escape no longer influences a cleartext-hop + # decision (decision 5). The per-connection declaration is what crosses it now — loudly, and + # recorded in the audit trail, instead of a process-wide env var nobody sees in review. + monkeypatch.delenv("MEFOR_ALLOW_INSECURE_TLS", raising=False) + with active_hop_posture(HopPosture(is_phi=True, enforcing=True)): dest = build_destination( Destination( name="OB", type=ConnectorType.DICOMWEB, settings=DICOMweb(url="http://pacs.example.org/dicom-web").settings, + cleartext_accepted=True, + cleartext_reason="legacy partner endpoint has no TLS", ) ) - assert isinstance(dest, DicomWebDestination) # built (warns loudly), not refused + assert isinstance(dest, DicomWebDestination) # built (warns loudly + audits), not refused # --- multipart framing ------------------------------------------------------- diff --git a/tests/test_docs_security_pathways.py b/tests/test_docs_security_pathways.py index e21997e2..18bd3936 100644 --- a/tests/test_docs_security_pathways.py +++ b/tests/test_docs_security_pathways.py @@ -222,8 +222,8 @@ def test_companion_table_covers_the_remaining_strength_dimensions() -> None: ( AuthSettings, "ad_session_recheck_seconds", - 0, - "`[auth].ad_session_recheck_seconds` (default **0 = off**)", + 300, + "`[auth].ad_session_recheck_seconds` (default **300 s**)", ), ( AuthSettings, diff --git a/tests/test_email_destination.py b/tests/test_email_destination.py index 3467952e..45e1f9e3 100644 --- a/tests/test_email_destination.py +++ b/tests/test_email_destination.py @@ -224,7 +224,12 @@ async def test_cleartext_send_path_when_escaped(monkeypatch: pytest.MonkeyPatch) def _cleartext_dest( - *, host: str = "smtp.partner.org", attested: bool = False, reason: str | None = None + *, + host: str = "smtp.partner.org", + attested: bool = False, + reason: str | None = None, + accepted: bool = False, + accept_reason: str | None = None, ) -> Destination: return Destination( name="OB_EMAIL", @@ -237,6 +242,8 @@ def _cleartext_dest( }, tls_hop_attested=attested, tls_hop_attested_reason=reason, + cleartext_accepted=accepted, + cleartext_reason=accept_reason, ) @@ -274,12 +281,26 @@ def test_staging_phi_cleartext_smtp_warns_and_crosses(monkeypatch: pytest.Monkey assert d._hop_guard is not None -def test_synthetic_cleartext_smtp_allowed(monkeypatch: pytest.MonkeyPatch) -> None: - # No PHI rides the hop → ALLOW, silently (the historical explicit-opt-in escape still applies). - monkeypatch.setenv(INSECURE_TLS_ESCAPE_ENV, "1") - with active_hop_posture(SYNTHETIC): - d = EmailDestination(_cleartext_dest()) +def test_cleartext_smtp_crosses_on_a_declaration(monkeypatch: pytest.MonkeyPatch) -> None: + """ADR 0153: the declaration crosses an ENFORCING cleartext SMTP hop; the data label no longer does. + + It also has to satisfy EmailDestination's own pre-gate, which fires BEFORE the shared authority — a + declaration that passed the authority but not the pre-gate would be dead config on SMTP alone.""" + monkeypatch.delenv(INSECURE_TLS_ESCAPE_ENV, raising=False) + with active_hop_posture(PROD_PHI): + d = EmailDestination( + _cleartext_dest(accepted=True, accept_reason="legacy relay has no STARTTLS") + ) assert d.use_tls is False + assert d._hop_guard is not None # crossed under a WARN, still guarded at send + + +def test_synthetic_cleartext_smtp_now_refused(monkeypatch: pytest.MonkeyPatch) -> None: + # SYNTHETIC here is ENFORCING. Pre-0153 the label alone allowed this hop; now only `enforcing` + # reaches the authority, so it refuses — and the escape cannot rescue it either (decision 5). + monkeypatch.delenv(INSECURE_TLS_ESCAPE_ENV, raising=False) + with active_hop_posture(SYNTHETIC), pytest.raises(ValueError, match="cleartext"): + EmailDestination(_cleartext_dest()) def test_unstamped_posture_is_byte_identical(monkeypatch: pytest.MonkeyPatch) -> None: @@ -300,10 +321,13 @@ async def test_send_time_backstop_refuses_at_the_byte_crossing( ) -> None: # Defense in depth: a reload that flips the instance to enforcing production-PHI must not put the # body on the wire just because construction happened under a laxer posture. - monkeypatch.setenv(INSECURE_TLS_ESCAPE_ENV, "1") + monkeypatch.delenv(INSECURE_TLS_ESCAPE_ENV, raising=False) _install_fake(monkeypatch) + # Constructed under a declaration (ADR 0153) rather than the retired synthetic carve-out. with active_hop_posture(SYNTHETIC): - d = EmailDestination(_cleartext_dest()) + d = EmailDestination( + _cleartext_dest(accepted=True, accept_reason="legacy relay has no STARTTLS") + ) assert d._hop_guard is not None d._hop_guard = InsecureHopGuard( host=d.host, diff --git a/tests/test_fhir_transport.py b/tests/test_fhir_transport.py index a22ebf8a..903e77f7 100644 --- a/tests/test_fhir_transport.py +++ b/tests/test_fhir_transport.py @@ -122,20 +122,24 @@ def test_fhir_cleartext_http_loopback_allowed() -> None: assert isinstance(dest, FhirDestination) -def test_fhir_cleartext_http_nonloopback_allowed_with_escape( +def test_fhir_cleartext_http_nonloopback_allowed_when_accepted( monkeypatch: pytest.MonkeyPatch, ) -> None: - monkeypatch.setenv("MEFOR_ALLOW_INSECURE_TLS", "1") - # #200 (ADR 0092): the escape downgrades REFUSE→WARN only on a NON-production instance (decision 2). - with active_hop_posture(HopPosture(is_phi=True, enforcing=False)): + # ADR 0153: the blunt MEFOR_ALLOW_INSECURE_TLS escape no longer influences a cleartext-hop + # decision (decision 5). The per-connection declaration is what crosses it now — loudly, and + # recorded in the audit trail, instead of a process-wide env var nobody sees in review. + monkeypatch.delenv("MEFOR_ALLOW_INSECURE_TLS", raising=False) + with active_hop_posture(HopPosture(is_phi=True, enforcing=True)): dest = build_destination( Destination( name="OB", type=ConnectorType.FHIR, settings=FHIR(url="http://fhir.example.org/fhir").settings, + cleartext_accepted=True, + cleartext_reason="legacy partner endpoint has no TLS", ) ) - assert isinstance(dest, FhirDestination) # built (warns loudly), not refused + assert isinstance(dest, FhirDestination) # built (warns loudly + audits), not refused def test_fhir_rejects_xml_format() -> None: diff --git a/tests/test_hop_refusal_db_inbound.py b/tests/test_hop_refusal_db_inbound.py index 1d752c1c..57a8ede4 100644 --- a/tests/test_hop_refusal_db_inbound.py +++ b/tests/test_hop_refusal_db_inbound.py @@ -199,7 +199,13 @@ def _src(conn: ConnectorType, *, host: str = "0.0.0.0", tls: bool = False, attes settings: dict[str, object] = {"host": host} if tls: settings["tls"] = True - return Source(type=conn, settings=settings, tls_hop_attested=attested) + return Source( + type=conn, + settings=settings, + tls_hop_attested=attested, + # ADR 0153 retro-fitted flag-implies-reason onto the attestation pair. + tls_hop_attested_reason="dedicated management VLAN" if attested else None, + ) _INBOUND_CHECKS = { diff --git a/tests/test_hop_refusal_http.py b/tests/test_hop_refusal_http.py index 0881bbb1..c2ff8220 100644 --- a/tests/test_hop_refusal_http.py +++ b/tests/test_hop_refusal_http.py @@ -6,11 +6,14 @@ which shipped escape-only, construction-only cleartext/verify-off refusals and are re-keyed onto the ONE authority (``tls_policy.insecure_hop_disposition``). Verifies, per cell: -* the posture gradient at CONSTRUCTION — production PHI REFUSES, a per-hop attestation / on-box loopback / - synthetic instance ALLOWs, the clamped global escape only downgrades a NON-prod PHI hop to WARN; -* **decision 5 (no-loosen)** — a *staging* (non-prod) PHI cleartext hop that refused today STILL refuses, - the gradient's non-prod WARN is floored back to REFUSE unless the escape/attestation applies; -* **decision 2 (escape clamp)** — ``MEFOR_ALLOW_INSECURE_TLS`` is inert on production; +* the gradient at CONSTRUCTION — an enforcing cleartext hop REFUSES, a per-hop attestation / on-box + loopback ALLOWs, and (ADR 0153) a per-connection ``cleartext_accepted`` declaration crosses it with a + loud, audited WARN; +* **ADR 0153 decision 1** — the data label is gone: a ``synthetic`` instance no longer gets a blanket + cleartext carve-out on any of these cells; +* **decision 5 (no-loosen)** — a hop that reaches WARN only via the non-enforcing dial is still floored + back to REFUSE by ``_shipped_strict_disposition`` unless it carries a declaration; +* **ADR 0153 decision 5** — ``MEFOR_ALLOW_INSECURE_TLS`` can no longer influence any of these decisions; * the fail-closed default — an UNSTAMPED posture is treated as production PHI; * the zero-I/O SEND-TIME re-assertion (decision 4) fires at the byte-crossing, before any wire I/O. @@ -70,6 +73,7 @@ def _build( spec: tuple[Any, Any, str], *, attested: bool = False, + accepted: bool = False, revocation_attested: bool = False, **over: Any, ) -> object: @@ -81,6 +85,12 @@ def _build( type=ctype, settings=settings, tls_hop_attested=attested, + # ADR 0153 retro-fitted flag-implies-reason onto the attestation, so supply one whenever the + # flag is set rather than leaving the pair incoherent at load. + tls_hop_attested_reason="proxy-terminated trusted segment" if attested else None, + # ADR 0153 decision 2: the opposite declaration — this hop is NOT secure and we accept that. + cleartext_accepted=accepted, + cleartext_reason="vendor firmware predates TLS" if accepted else None, # #201 (ADR 0078 amendment): a VERIFYING https hop now also carries a revocation refusal; # tests that build a prod-PHI verified remote hop and expect SUCCESS attest revocation here # so the (distinct) cleartext #200 assertions stay the subject under test. @@ -115,14 +125,20 @@ def test_cleartext_prod_phi_refuses(cell: str, monkeypatch: pytest.MonkeyPatch) def test_escape_inert_on_production(cell: str, monkeypatch: pytest.MonkeyPatch) -> None: monkeypatch.setenv("MEFOR_ALLOW_INSECURE_TLS", "1") with active_hop_posture(_PROD), pytest.raises(InsecureHopRefused): - _build(_CLEARTEXT[cell]) # escape cannot satisfy a prod-PHI hop + _build(_CLEARTEXT[cell]) # escape cannot satisfy an enforcing hop @pytest.mark.parametrize("cell", _CELLS) -def test_escape_downgrades_on_non_prod(cell: str, monkeypatch: pytest.MonkeyPatch) -> None: +def test_escape_is_inert_on_non_prod_too(cell: str, monkeypatch: pytest.MonkeyPatch) -> None: + """ADR 0153 decision 5, for the HTTP family: the env var no longer relaxes ANY cleartext hop. + + This asserted the opposite before 0153 (the escape downgraded a non-prod hop to a crossing WARN). + ``_shipped_strict_disposition``'s no-loosen floor is now keyed on ``cleartext_accepted`` instead of + on the escape, which is what makes decision 2's per-connection declaration effective here — and what + stops the blunt env var being alive on HTTP while dead on raw TCP. It is a deliberate TIGHTENING.""" monkeypatch.setenv("MEFOR_ALLOW_INSECURE_TLS", "1") - with active_hop_posture(_STAGING): - assert _build(_CLEARTEXT[cell]) is not None # warns-and-builds, not refused + with active_hop_posture(_STAGING), pytest.raises(InsecureHopRefused): + _build(_CLEARTEXT[cell]) # --- the ALLOW arms: loopback, per-hop attestation, synthetic (non-PHI) ------------------------------- @@ -145,11 +161,47 @@ def test_attestation_allows_prod_phi_cleartext(cell: str, monkeypatch: pytest.Mo @pytest.mark.parametrize("cell", _CELLS) -def test_synthetic_instance_allows_cleartext(cell: str, monkeypatch: pytest.MonkeyPatch) -> None: - # No PHI rides the hop → allowed (the gradient's arm 3), even off-loopback with no escape. +def test_synthetic_instance_no_longer_allows_cleartext( + cell: str, monkeypatch: pytest.MonkeyPatch +) -> None: + """ADR 0153 decision 1 on the HTTP family: no data label may permit a cleartext hop. + + This asserted ALLOW before 0153 — and note the posture here is non-enforcing too, so BOTH of the + old relaxations are gone: the label no longer allows, and the non-enforcing WARN is floored.""" monkeypatch.delenv("MEFOR_ALLOW_INSECURE_TLS", raising=False) - with active_hop_posture(_SYNTHETIC): - assert _build(_CLEARTEXT[cell]) is not None + with active_hop_posture(_SYNTHETIC), pytest.raises(InsecureHopRefused): + _build(_CLEARTEXT[cell]) + + +@pytest.mark.parametrize("cell", _CELLS) +def test_cleartext_accepted_crosses_an_enforcing_http_hop( + cell: str, monkeypatch: pytest.MonkeyPatch +) -> None: + """ADR 0153 decision 2 on the HTTP family — the largest cleartext-egress family in the product. + + This is the case ``_shipped_strict_disposition``'s re-keyed floor exists for: without it the + declaration would be silently inert for REST/SOAP/FHIR/DICOMweb, i.e. everywhere it is a genuine + escape (Tcp()/X12() never reach that cell).""" + monkeypatch.delenv("MEFOR_ALLOW_INSECURE_TLS", raising=False) + with active_hop_posture(_PROD): + assert _build(_CLEARTEXT[cell], accepted=True) is not None + + +@pytest.mark.parametrize("cell", _CELLS) +def test_cleartext_accepted_is_audited_at_every_construction( + cell: str, monkeypatch: pytest.MonkeyPatch, caplog: pytest.LogCaptureFixture +) -> None: + """WARN + a DEDICATED audit record, so an accepted risk cannot quietly stop being visible.""" + monkeypatch.delenv("MEFOR_ALLOW_INSECURE_TLS", raising=False) + with active_hop_posture(_PROD), caplog.at_level("WARNING"): + _build(_CLEARTEXT[cell], accepted=True) + audit = [ + r.getMessage() + for r in caplog.records + if "cleartext hop crossed on an operator acceptance" in r.getMessage() + ] + assert audit, "an accepted cleartext hop must produce an audit record at every construction" + assert "vendor firmware predates TLS" in audit[0] # --- fail-closed: an UNSTAMPED posture is treated as production PHI ----------------------------------- @@ -262,6 +314,32 @@ def test_fhir_lookup_attested_read_allowed_on_prod(monkeypatch: pytest.MonkeyPat assert ex.connections == frozenset({"L"}) +def test_fhir_lookup_cleartext_accepted_read_allowed_on_prod( + monkeypatch: pytest.MonkeyPatch, +) -> None: + """A FhirLookup connection has no Destination, so its declaration is a ``FhirLookup()`` parameter. + + Driven through the FACTORY, not a hand-built settings dict: a settings key an operator has no way + to write would be an escape that exists only in the test, unvalidated and invisible to the + loosening registry. Without this the read path would be the one cleartext-egress cell with no + expressible declaration at all.""" + from messagefoundry.config import wiring + from messagefoundry.config.wiring import FhirLookup, Registry + + monkeypatch.delenv("MEFOR_ALLOW_INSECURE_TLS", raising=False) + reg = Registry() + monkeypatch.setattr(wiring, "_active", reg) + FhirLookup( + "L", + url="http://fhir.example.org/fhir", + cleartext_accepted=True, + cleartext_reason="legacy on-prem FHIR facade has no TLS", + ) + with active_hop_posture(_PROD): + ex = FhirLookupExecutor({"L": reg.fhir_lookups["L"].settings}) + assert ex.connections == frozenset({"L"}) + + # --- the send-time guard object (zero-I/O re-assertion, decision 4) ----------------------------------- @@ -272,16 +350,18 @@ def test_send_guard_refuses_prod_phi_hop(monkeypatch: pytest.MonkeyPatch) -> Non guard.assert_send("api.example.com", "http://api.example.com/x") -def test_send_guard_allows_attested_and_synthetic_and_loopback( +def test_send_guard_allows_attested_accepted_and_loopback( monkeypatch: pytest.MonkeyPatch, ) -> None: monkeypatch.delenv("MEFOR_ALLOW_INSECURE_TLS", raising=False) - # attested → ALLOW even on prod PHI + # attested → ALLOW even on an enforcing instance InsecureHopGuard(posture=_PROD, attested=True, cell="c").assert_send( "api.example.com", "http://api.example.com/x" ) - # synthetic → ALLOW - InsecureHopGuard(posture=_SYNTHETIC, attested=False, cell="c").assert_send( + # declared cleartext_accepted → WARN (crosses) rather than REFUSE, so the send is permitted. The + # send-time guard must carry the SAME declaration the construction gate decided on, or a permitted + # connection would blow up at the byte-crossing instead. + InsecureHopGuard(posture=_PROD, attested=False, cell="c", cleartext_accepted=True).assert_send( "api.example.com", "http://api.example.com/x" ) # loopback host → ALLOW (on-box) @@ -290,16 +370,32 @@ def test_send_guard_allows_attested_and_synthetic_and_loopback( ) +def test_send_guard_refuses_a_synthetic_instance_now(monkeypatch: pytest.MonkeyPatch) -> None: + """The send-time backstop tracks the authority: no data label crosses it either (ADR 0153).""" + monkeypatch.delenv("MEFOR_ALLOW_INSECURE_TLS", raising=False) + guard = InsecureHopGuard(posture=_SYNTHETIC, attested=False, cell="c") + with pytest.raises(InsecureHopRefused): + guard.assert_send("api.example.com", "http://api.example.com/x") + + # --- helper-level: refuse_cleartext_egress / refuse_verify_off return a guard only when PERMITTED ----- def test_refuse_cleartext_egress_returns_guard_when_permitted( monkeypatch: pytest.MonkeyPatch, ) -> None: - monkeypatch.setenv("MEFOR_ALLOW_INSECURE_TLS", "1") + # Permitted now means DECLARED (the env escape no longer permits anything here) — and the returned + # guard must carry the declaration forward so the send-time re-assertion agrees with construction. + monkeypatch.delenv("MEFOR_ALLOW_INSECURE_TLS", raising=False) with active_hop_posture(_STAGING): - guard = refuse_cleartext_egress("http", "http://api.example.com/x") + guard = refuse_cleartext_egress( + "http", + "http://api.example.com/x", + cleartext_accepted=True, + cleartext_reason="legacy peer", + ) assert isinstance(guard, InsecureHopGuard) + assert guard.cleartext_accepted is True def test_refuse_cleartext_egress_no_guard_for_loopback_or_https( @@ -357,19 +453,27 @@ def open(self, req: object, timeout: float | None = None) -> _Resp: async def test_rest_send_time_assertion_blocks_before_wire( monkeypatch: pytest.MonkeyPatch, ) -> None: - # Build a permitted cleartext REST dest (staging + escape → WARN), then revoke the escape: the - # zero-I/O send-time re-assertion must REFUSE at the byte-crossing, before the opener is ever called. - monkeypatch.setenv("MEFOR_ALLOW_INSECURE_TLS", "1") - with active_hop_posture(_STAGING): - dest = _build(_CLEARTEXT["REST"]) + # Build a PERMITTED cleartext REST dest (an enforcing instance + a cleartext_accepted declaration → + # WARN), then swap in a guard whose captured state REFUSES: the zero-I/O send-time re-assertion must + # fire at the byte-crossing, before the opener is ever called. + # + # Pre-ADR-0153 this test revoked MEFOR_ALLOW_INSECURE_TLS between the two sends. That lever no longer + # exists (decision 5 unhooked it), and swapping the guard is the better test of 0092 decision 4 + # anyway: it exercises the re-assertion itself rather than a since-removed env read inside it. + monkeypatch.delenv("MEFOR_ALLOW_INSECURE_TLS", raising=False) + with active_hop_posture(_PROD): + dest = _build(_CLEARTEXT["REST"], accepted=True) opener = _Opener() dest._opener = opener # type: ignore[attr-defined] assert dest._hop_guard is not None # type: ignore[attr-defined] - await dest.send("payload") # escape still set → permitted, reaches the (fake) wire + await dest.send("payload") # declared → permitted, reaches the (fake) wire assert opener.calls == 1 - monkeypatch.delenv("MEFOR_ALLOW_INSECURE_TLS", raising=False) + # A reload / per-message re-target could leave a guard whose decision is now REFUSE. + dest._hop_guard = InsecureHopGuard( # type: ignore[attr-defined] + posture=_PROD, attested=False, cell="HTTP cleartext egress", cleartext_accepted=False + ) with pytest.raises(InsecureHopRefused): - await dest.send("payload") # escape revoked → send-time refusal, before any I/O + await dest.send("payload") # send-time refusal, before any I/O assert opener.calls == 1 # the opener was NOT called on the refused send diff --git a/tests/test_hop_refusal_rawtcp.py b/tests/test_hop_refusal_rawtcp.py index 2cf8f642..38d2dce3 100644 --- a/tests/test_hop_refusal_rawtcp.py +++ b/tests/test_hop_refusal_rawtcp.py @@ -13,10 +13,13 @@ cleartext hop off-loopback; and * a zero-I/O send-time backstop re-asserts it at the byte crossing. -Loopback / synthetic / per-connection-attested hops ALLOW (byte-identical); a non-production PHI hop -WARNs (crosses, loud-logged); the global escape may only downgrade REFUSE→WARN on non-production. A hop -built OUTSIDE the stamped gate (a live serve build after the pre-flight, or a direct test/embedding — -posture unstamped) is a no-op, so no existing lane breaks. +**Amended by ADR 0153**: the authority no longer reads the data label, so a ``synthetic`` instance is +NOT carved out any more — it refuses exactly like a PHI one. Loopback and per-connection-attested hops +still ALLOW (byte-identical); a non-enforcing instance WARNs (crosses, loud-logged); a per-connection +``cleartext_accepted`` declaration WARNs + audits even under ENFORCE; and ``MEFOR_ALLOW_INSECURE_TLS`` +can no longer influence the decision at all. A hop built OUTSIDE the stamped gate (a live serve build +after the pre-flight, or a direct test/embedding — posture unstamped) is still a no-op, so no existing +lane breaks. """ from __future__ import annotations @@ -37,10 +40,12 @@ from messagefoundry.transports.tcp import TcpDestination from messagefoundry.transports.x12 import X12Destination -# The three postures the gradient keys on (the AI-derived is_phi/production). +# The three postures. Only `enforcing` reaches the cleartext authority now (ADR 0153) — `is_phi` is +# retained on HopPosture for the revocation / inbound-bind / verify-off gates, which still read it. PROD_PHI = HopPosture(is_phi=True, enforcing=True) STAGING_PHI = HopPosture(is_phi=True, enforcing=False) -SYNTHETIC = HopPosture(is_phi=False, enforcing=True) # not is_phi → always ALLOW +# Pre-0153 this was the blanket carve-out ("not is_phi → ALLOW"). It is enforcing, so it now REFUSES. +SYNTHETIC = HopPosture(is_phi=False, enforcing=True) REMOTE = "10.0.0.5" # a non-loopback host (never resolves; treated as remote/off-box) LOOPBACK = "127.0.0.1" @@ -49,12 +54,39 @@ # --- config builders (one plaintext outbound per transport) ------------------ +# ADR 0153 retro-fitted "the flag REQUIRES a reason" onto tls_hop_attested, so every builder below +# supplies a default reason whenever its flag is set rather than leaving the pair incoherent at load. +_ATTEST_REASON = "proxy-terminated trusted segment" +_ACCEPT_REASON = "vendor firmware predates TLS; segment is not isolated" + + +def _hop_fields( + attested: bool, reason: str | None, accepted: bool, accept_reason: str | None +) -> dict[str, object]: + """The four hop-declaration fields every Destination builder below shares. + + ``reason``/``accept_reason`` fall back to a non-empty string whenever their flag is set, because + BOTH pairs are now load-validated as flag-implies-reason (ADR 0153 decision 2).""" + return { + "tls_hop_attested": attested, + "tls_hop_attested_reason": ( + reason if reason is not None or not attested else _ATTEST_REASON + ), + "cleartext_accepted": accepted, + "cleartext_reason": ( + accept_reason if accept_reason is not None or not accepted else _ACCEPT_REASON + ), + } + + def mllp_cfg( host: str, *, tls: bool = False, attested: bool = False, reason: str | None = None, + accepted: bool = False, + accept_reason: str | None = None, revocation_attested: bool = False, ) -> Destination: settings: dict[str, object] = {"host": host, "port": 5000} @@ -64,34 +96,51 @@ def mllp_cfg( name="OB_MLLP", type=ConnectorType.MLLP, settings=settings, - tls_hop_attested=attested, - tls_hop_attested_reason=reason, tls_revocation_attested=revocation_attested, + **_hop_fields(attested, reason, accepted, accept_reason), ) -def tcp_cfg(host: str, *, attested: bool = False, reason: str | None = None) -> Destination: +def tcp_cfg( + host: str, + *, + attested: bool = False, + reason: str | None = None, + accepted: bool = False, + accept_reason: str | None = None, +) -> Destination: return Destination( name="OB_TCP", type=ConnectorType.TCP, settings={"host": host, "port": 5000, "framing": "stx_etx"}, - tls_hop_attested=attested, - tls_hop_attested_reason=reason, + **_hop_fields(attested, reason, accepted, accept_reason), ) -def x12_cfg(host: str, *, attested: bool = False, reason: str | None = None) -> Destination: +def x12_cfg( + host: str, + *, + attested: bool = False, + reason: str | None = None, + accepted: bool = False, + accept_reason: str | None = None, +) -> Destination: return Destination( name="OB_X12", type=ConnectorType.X12, settings={"host": host, "port": 5000}, - tls_hop_attested=attested, - tls_hop_attested_reason=reason, + **_hop_fields(attested, reason, accepted, accept_reason), ) def dicom_cfg( - host: str, *, tls: bool = False, attested: bool = False, reason: str | None = None + host: str, + *, + tls: bool = False, + attested: bool = False, + reason: str | None = None, + accepted: bool = False, + accept_reason: str | None = None, ) -> Destination: settings: dict[str, object] = {"ae_title": "MF_SCU", "host": host, "port": 104} if tls: @@ -100,24 +149,31 @@ def dicom_cfg( name="OB_DICOM", type=ConnectorType.DIMSE, settings=settings, - tls_hop_attested=attested, - tls_hop_attested_reason=reason, + **_hop_fields(attested, reason, accepted, accept_reason), ) def ftp_cfg( - host: str, *, attested: bool = False, reason: str | None = None, protocol: str = "ftp" + host: str, + *, + attested: bool = False, + reason: str | None = None, + accepted: bool = False, + accept_reason: str | None = None, + protocol: str = "ftp", ) -> Destination: settings: dict[str, object] = { "host": host, "remote_dir": "/in", "protocol": protocol, } + fields = _hop_fields(attested, reason, accepted, accept_reason) if attested: + # The anonymous-ftp guard reads the ATTESTATION off the settings mapping (that is how the + # runner threads it for this transport), so mirror it there exactly as _dest_config does. settings["tls_hop_attested"] = True - if reason is not None: - settings["tls_hop_attested_reason"] = reason - return Destination(name="OB_FTP", type=ConnectorType.REMOTEFILE, settings=settings) + settings["tls_hop_attested_reason"] = fields["tls_hop_attested_reason"] + return Destination(name="OB_FTP", type=ConnectorType.REMOTEFILE, settings=settings, **fields) # Each entry builds ONE plaintext outbound connector from a config-builder; used to run the whole @@ -151,9 +207,14 @@ def test_construction_allows_loopback(build_cfg, connector) -> None: @pytest.mark.parametrize(("build_cfg", "connector"), PLAINTEXT_BUILDERS) -def test_construction_allows_synthetic_instance(build_cfg, connector) -> None: - with active_hop_posture(SYNTHETIC): - connector(build_cfg(REMOTE)) # not is_phi → no PHI on the wire → ALLOW +def test_construction_refuses_synthetic_instance(build_cfg, connector) -> None: + """ADR 0153's headline: a ``synthetic`` data label no longer buys a cleartext hop. + + This test asserted ALLOW before 0153. The label is authored in the same file as the hosts it + governs, by the same hand, and a typo in it was indistinguishable from a declaration — with every + transport hop in the product as its blast radius. The instance is enforcing, so it now refuses.""" + with active_hop_posture(SYNTHETIC), pytest.raises(InsecureHopRefused): + connector(build_cfg(REMOTE)) # --- construction gate: WARN (cross) a non-production PHI hop ----------------- @@ -207,15 +268,102 @@ def test_dicom_tls_has_no_hop_guard() -> None: # --- escape clamp: MEFOR_ALLOW_INSECURE_TLS only downgrades on NON-production -- -def test_escape_downgrades_staging_phi_but_never_prod(monkeypatch) -> None: +def test_global_escape_can_no_longer_influence_a_cleartext_hop(monkeypatch) -> None: + """ADR 0153 decision 5: the variable is UNHOOKED from this authority (not deleted). + + Asserted in BOTH directions — that it does not relax an enforcing hop, and that it does not change + the non-enforcing one either. A one-directional check would still pass if the escape were quietly + re-hooked as a WARN arm, which is exactly the regression the decision forbids.""" + with active_hop_posture(STAGING_PHI): + TcpDestination(tcp_cfg(REMOTE)) # non-enforcing → WARN, escape unset + with active_hop_posture(PROD_PHI), pytest.raises(InsecureHopRefused): + TcpDestination(tcp_cfg(REMOTE)) # enforcing → REFUSE, escape unset monkeypatch.setenv(INSECURE_TLS_ESCAPE_ENV, "1") - # Non-production PHI is already WARN; the escape keeps it crossing (still no refusal). with active_hop_posture(STAGING_PHI): - TcpDestination(tcp_cfg(REMOTE)) - # Production PHI: the escape is CLAMPED to non-production, so it can NEVER satisfy a prod-PHI hop — - # the production REFUSE arm still wins even with the escape set (decision 2 behaviour change). + TcpDestination(tcp_cfg(REMOTE)) # unchanged by the escape with active_hop_posture(PROD_PHI), pytest.raises(InsecureHopRefused): - TcpDestination(tcp_cfg(REMOTE)) + TcpDestination(tcp_cfg(REMOTE)) # STILL refuses with the escape set + + +# --- ADR 0153 decision 2: the honest per-connection acceptance ---------------- + + +@pytest.mark.parametrize(("build_cfg", "connector"), PLAINTEXT_BUILDERS) +def test_cleartext_accepted_crosses_an_enforcing_hop_and_is_audited( + build_cfg, connector, caplog +) -> None: + """It yields WARN, never ALLOW — crossed, but logged AND audited at every construction.""" + with active_hop_posture(PROD_PHI), caplog.at_level("WARNING"): + connector(build_cfg(REMOTE, accepted=True, accept_reason="vendor firmware predates TLS")) + # The generic WARN half... + assert any("insecure transport hop permitted" in r.message for r in caplog.records) + # ...and the DEDICATED audit record naming the declaration + its reason, which is what tells an + # auditor this hop was crossed on an operator's accepted risk rather than on a warn-mode dial. + audit = [ + r for r in caplog.records if "cleartext hop crossed on an operator acceptance" in r.message + ] + assert audit, "an accepted cleartext hop must produce an audit record at every construction" + assert "vendor firmware predates TLS" in audit[0].getMessage() + + +@pytest.mark.parametrize(("build_cfg", "connector"), PLAINTEXT_BUILDERS) +def test_the_acceptance_record_names_the_declaring_connection(build_cfg, connector, caplog) -> None: + """The record must lead back to the LINE TO FIX, or it is not much of a record. + + ``cell`` is a static family label ("TCP outbound", "HTTP cleartext egress") and the HTTP-family + message carries a host with no port — so with two destinations to the same host an auditor could + not tell which declaration produced the crossing. docs/SECURITY-LOOSENING.md and docs/CONNECTIONS.md + both promise the connection name; this is what makes that true.""" + cfg = build_cfg(REMOTE, accepted=True, accept_reason="vendor firmware predates TLS") + with active_hop_posture(PROD_PHI), caplog.at_level("WARNING"): + connector(cfg) + audit = [ + r.getMessage() + for r in caplog.records + if "cleartext hop crossed on an operator acceptance" in r.message + ] + assert audit + assert cfg.name in audit[0], f"the acceptance record must name {cfg.name!r}: {audit[0]}" + + +def test_acceptance_and_attestation_are_distinguishable_in_the_trail(caplog) -> None: + """The one distinction the audit trail exists to preserve (ADR 0153 decision 2's table). + + ``tls_hop_attested`` = "this hop IS secure by means the engine cannot see" → ALLOW, and its record + says *attestation*. ``cleartext_accepted`` = "this hop is NOT secure and we accept that" → WARN, and + its record says *accepted*. Collapsing the two would leave the trail unable to tell a + proxy-terminated hop from plaintext PHI on a flat network.""" + with active_hop_posture(PROD_PHI), caplog.at_level("WARNING"): + TcpDestination(tcp_cfg(REMOTE, attested=True, reason="proxy-terminated trusted segment")) + attested_records = [r.getMessage() for r in caplog.records] + assert any("operator attestation" in m for m in attested_records) + assert not any("cleartext hop crossed on an operator acceptance" in m for m in attested_records) + + caplog.clear() + with active_hop_posture(PROD_PHI), caplog.at_level("WARNING"): + TcpDestination(tcp_cfg(REMOTE, accepted=True, accept_reason="no TLS on this device")) + accepted_records = [r.getMessage() for r in caplog.records] + assert any("cleartext hop crossed on an operator acceptance" in m for m in accepted_records) + assert not any("operator attestation" in m for m in accepted_records) + + +def test_tcp_and_x12_acceptance_is_permanent_not_transitional() -> None: + """ADR 0153 decision 4: Tcp()/X12() have NO TLS support, so there is no `tls = true` to migrate to. + + Pinned as a test so a future reader cannot mistake the standing declaration for unfinished + migration work: neither factory accepts a `tls` argument (BACKLOG #311 tracks adding one).""" + import inspect + + from messagefoundry.config.wiring import X12, Tcp + + assert "tls" not in inspect.signature(Tcp).parameters + assert "tls" not in inspect.signature(X12).parameters + # ...and without the declaration the hop simply refuses under ENFORCE — there is no other escape + # short of an attestation, which would be a false statement about a plaintext-only transport. + with active_hop_posture(PROD_PHI), pytest.raises(InsecureHopRefused): + X12Destination(x12_cfg(REMOTE)) + with active_hop_posture(PROD_PHI): + X12Destination(x12_cfg(REMOTE, accepted=True, accept_reason="X12 VAN link has no TLS")) # --- send-time backstop (zero-I/O, before the first payload byte) ------------- @@ -297,7 +445,9 @@ def test_assert_send_matrix() -> None: def test_guard_disposition_precedence() -> None: - def disp(host: str, *, posture: HopPosture, attested: bool) -> HopDisposition: + def disp( + host: str, *, posture: HopPosture, attested: bool, accepted: bool = False + ) -> HopDisposition: g = InsecureHopGuard( host=host, port=1, @@ -306,14 +456,19 @@ def disp(host: str, *, posture: HopPosture, attested: bool) -> HopDisposition: attested=attested, attested_reason=None, posture=posture, + cleartext_accepted=accepted, ) return g._disposition(posture) assert disp(LOOPBACK, posture=PROD_PHI, attested=False) is HopDisposition.ALLOW assert disp(REMOTE, posture=PROD_PHI, attested=True) is HopDisposition.ALLOW - assert disp(REMOTE, posture=SYNTHETIC, attested=False) is HopDisposition.ALLOW + # ADR 0153: the synthetic carve-out is GONE — an enforcing instance refuses whatever its label. + assert disp(REMOTE, posture=SYNTHETIC, attested=False) is HopDisposition.REFUSE assert disp(REMOTE, posture=PROD_PHI, attested=False) is HopDisposition.REFUSE assert disp(REMOTE, posture=STAGING_PHI, attested=False) is HopDisposition.WARN + # ...and the new arm 3: an acceptance WARNs (crosses) even under ENFORCE, and never ALLOWs. + assert disp(REMOTE, posture=PROD_PHI, attested=False, accepted=True) is HopDisposition.WARN + assert disp(LOOPBACK, posture=PROD_PHI, attested=False, accepted=True) is HopDisposition.ALLOW # --- anonymous-ftp guard: encrypted / credentialed protocols carry no guard --- diff --git a/tests/test_hop_refusal_residuals.py b/tests/test_hop_refusal_residuals.py index 0661cc96..1af78e4c 100644 --- a/tests/test_hop_refusal_residuals.py +++ b/tests/test_hop_refusal_residuals.py @@ -28,6 +28,7 @@ import pytest from messagefoundry.api import create_app +from messagefoundry.config import wiring from messagefoundry.config.settings import ( INSECURE_TLS_ESCAPE_ENV, AiSettings, @@ -40,7 +41,12 @@ InsecureHopRefused, phi_read_hop_disposition, ) -from messagefoundry.config.wiring import DatabaseLookupSpec, FhirLookupSpec, Registry +from messagefoundry.config.wiring import ( + DatabaseLookupSpec, + FhirLookup, + FhirLookupSpec, + Registry, +) from messagefoundry.pipeline import Engine from messagefoundry.pipeline.wiring_runner import RegistryRunner from messagefoundry.store import MessageStore @@ -194,9 +200,28 @@ async def store(tmp_path: Path): # type: ignore[no-untyped-def] await s.close() -def _fhir_registry() -> Registry: +def _fhir_registry(*, accepted: bool = False) -> Registry: + """Build the lookup graph through the REAL authoring surface when a declaration is involved. + + ADR 0153's pair is a ``FhirLookup()`` parameter, not a hand-written settings key: driving the + factory is what proves the surface an operator actually has works end to end (and that the + flag/reason coherence rule fires on it). The undeclared arm stays hand-built — there is nothing to + validate, and it keeps the refusal case independent of the factory.""" reg = Registry() - reg.add_fhir_lookup(FhirLookupSpec("epic", {"url": CLEARTEXT_FHIR})) + if not accepted: + reg.add_fhir_lookup(FhirLookupSpec("epic", {"url": CLEARTEXT_FHIR})) + return reg + prev = wiring._active + wiring._active = reg + try: + FhirLookup( + "epic", + url=CLEARTEXT_FHIR, + cleartext_accepted=True, + cleartext_reason="legacy on-prem FHIR facade has no TLS", + ) + finally: + wiring._active = prev return reg @@ -229,10 +254,25 @@ async def test_runner_refuses_prod_phi_cleartext_fhir_lookup(store: MessageStore runner._build_fhir_lookup_executor() -async def test_runner_allows_synthetic_cleartext_fhir_lookup(store: MessageStore) -> None: - # A synthetic instance is NOT false-closed: the same cleartext read builds fine. Before the residual - # the live-runner build was unstamped → fail-closed (prod-PHI) → this would have wrongly refused. +async def test_runner_refuses_synthetic_cleartext_fhir_lookup(store: MessageStore) -> None: + """ADR 0153: the synthetic label no longer buys the cleartext read hop either. + + The residual this test belongs to is about the live-runner build reading the STAMPED posture rather + than fail-closing — that is still what is under test. What changed is the expected answer for an + enforcing synthetic instance: the label is no longer an input, so it refuses.""" runner = RegistryRunner(_fhir_registry(), store, poll_interval=0.02, hop_posture=SYNTHETIC) + with pytest.raises(InsecureHopRefused): + runner._build_fhir_lookup_executor() + + +async def test_runner_allows_declared_cleartext_fhir_lookup(store: MessageStore) -> None: + """...and the live-runner build is NOT false-closed when the read hop carries a declaration. + + This is the half of the residual that still matters: an unstamped build would fail closed and + wrongly refuse a legitimately-declared lane at serve, after build_check had already allowed it.""" + runner = RegistryRunner( + _fhir_registry(accepted=True), store, poll_interval=0.02, hop_posture=PROD_PHI + ) assert runner._build_fhir_lookup_executor() is not None @@ -274,6 +314,30 @@ async def test_runner_allows_synthetic_weakened_db_lookup_with_escape( outbound("OB", Rest(url="http://collector.example.org/ingest")) +@router("r") +def route(msg): + return ["h"] + + +@handler("h") +def handle(msg): + return Send("OB", msg) +""" + +# The same graph with the ADR 0153 per-destination declaration, so the cleartext hop crosses with a +# loud audited WARN instead of refusing — the honest escape for a peer that cannot do TLS. +_CONFIG_MODULE_ACCEPTED = """ +from messagefoundry import MLLP, Rest, Send, handler, inbound, outbound, router + +inbound("IB", MLLP(port=15099), router="r") +outbound( + "OB", + Rest(url="http://collector.example.org/ingest"), + cleartext_accepted=True, + cleartext_reason="legacy collector predates TLS", +) + + @router("r") def route(msg): return ["h"] @@ -297,10 +361,13 @@ def handle(msg): """ -def _write_config(tmp_path: Path, *, env: str, synthetic: bool = False) -> Path: +def _write_config( + tmp_path: Path, *, env: str, synthetic: bool = False, accepted: bool = False +) -> Path: cfg = tmp_path / "config" cfg.mkdir() - (cfg / "feed.py").write_text(_CONFIG_MODULE, encoding="utf-8") + module = _CONFIG_MODULE_ACCEPTED if accepted else _CONFIG_MODULE + (cfg / "feed.py").write_text(module, encoding="utf-8") # GIVEN 1 (ADR 0148): dev derives PHI now, so a synthetic instance declares the opt-out explicitly. synthetic_line = "security.handles_real_patient_data = false\n" if synthetic else "" (tmp_path / "messagefoundry.toml").write_text( @@ -324,14 +391,40 @@ def test_check_build_refuses_prod_phi_cleartext_hop(tmp_path: Path) -> None: assert not report.ok # the whole gate fails on a blocking required check -def test_check_build_allows_dev_cleartext_hop(tmp_path: Path) -> None: +def test_check_build_refuses_dev_synthetic_cleartext_hop(tmp_path: Path) -> None: from messagefoundry.checks import run_checks cfg = _write_config(tmp_path, env="dev", synthetic=True) report = run_checks(cfg, run_lint=False) result = _build_result(report) - # A synthetic (dev) instance: the SAME cleartext hop is allowed — byte-identical, no false-close. - assert result.ok and not result.skipped + # ADR 0153: a synthetic dev instance no longer gets the SAME cleartext hop for free. This is the + # documented cost of the change ("a synthetic-data instance loses its blanket carve-out") — the + # remedy is a per-destination declaration, or [security].enforcement = warn. + assert result.required and not result.ok and not result.skipped + + +def test_check_build_allows_declared_cleartext_hop(tmp_path: Path) -> None: + from messagefoundry.checks import run_checks + + cfg = _write_config(tmp_path, env="prod", accepted=True) + report = run_checks(cfg, run_lint=False) + assert _build_result(report).ok + # ...and `check` SURFACES the whole accepted set, which is the mitigation ADR 0153 names for the + # risk that an operator declares the acceptance broadly enough to approximate the blanket escape. + surfaced = next(r for r in report.results if r.name == "cleartext-accepted") + assert surfaced.ok and not surfaced.required and not surfaced.skipped + assert "OB" in surfaced.detail and "legacy collector" in surfaced.detail + + +def test_check_cleartext_accepted_reports_an_empty_set(tmp_path: Path) -> None: + from messagefoundry.checks import run_checks + + # The surface must say "none" explicitly rather than go quiet: an absent line is indistinguishable + # from a check that did not run, which is how a green gate stops being evidence. + cfg = _write_config(tmp_path, env="prod") + report = run_checks(cfg, run_lint=False) + surfaced = next(r for r in report.results if r.name == "cleartext-accepted") + assert surfaced.ok and "no connection declares cleartext_accepted" in surfaced.detail def test_check_build_skips_without_service_toml(tmp_path: Path) -> None: diff --git a/tests/test_hop_refusal_revocation.py b/tests/test_hop_refusal_revocation.py index f1cf69e2..4b713ca1 100644 --- a/tests/test_hop_refusal_revocation.py +++ b/tests/test_hop_refusal_revocation.py @@ -176,10 +176,21 @@ def test_mllp_blanket_env_allows_prod_phi(monkeypatch: pytest.MonkeyPatch) -> No def test_mllp_sets_revocation_guard_only_on_verify_path() -> None: # verify-ON TLS hop carries a revocation guard; a cleartext (tls off) hop does not (its cleartext # #200 guard handles it — the two guards are disjoint, never both set). - with active_hop_posture(SYNTHETIC): # synthetic so neither guard refuses + # + # SYNTHETIC used to suppress BOTH gates. Since ADR 0153 the data label no longer relaxes the + # CLEARTEXT one, so the cleartext leg carries an explicit declaration instead. The revocation gate + # (ADR 0078) still reads the label and is deliberately OUT of 0153's scope — which is precisely what + # makes the asymmetry in this test the thing worth pinning. + with active_hop_posture(SYNTHETIC): verified = MLLPDestination(mllp_cfg(REMOTE)) cleartext = MLLPDestination( - Destination(name="OB", type=ConnectorType.MLLP, settings={"host": REMOTE, "port": 5000}) + Destination( + name="OB", + type=ConnectorType.MLLP, + settings={"host": REMOTE, "port": 5000}, + cleartext_accepted=True, + cleartext_reason="legacy peer has no TLS", + ) ) assert verified._revocation_guard is not None and verified._hop_guard is None assert cleartext._revocation_guard is None and cleartext._hop_guard is not None diff --git a/tests/test_hop_refusal_serve_clamp.py b/tests/test_hop_refusal_serve_clamp.py index 0433d87e..ac077548 100644 --- a/tests/test_hop_refusal_serve_clamp.py +++ b/tests/test_hop_refusal_serve_clamp.py @@ -37,6 +37,9 @@ PROD_PHI = HopPosture(is_phi=True, enforcing=True) STAGING_PHI = HopPosture(is_phi=True, enforcing=False) SYNTHETIC = HopPosture(is_phi=False, enforcing=False) # dev / synthetic instance (no PHI) +# ADR 0153: an ENFORCING instance that merely carries a synthetic label. Pre-0153 the label alone +# allowed every cleartext hop; now only `enforcing` reaches the authority, so this one refuses. +SYNTHETIC_ENFORCING = HopPosture(is_phi=False, enforcing=True) REMOTE = "10.0.0.5" # a non-loopback host (never resolves; treated as off-box) @@ -65,15 +68,24 @@ async def test_serve_refuses_prod_phi_cleartext_tcp_outbound(store: MessageStore await runner.stop() -async def test_serve_allows_synthetic_cleartext_http_outbound(store: MessageStore) -> None: - # Finding 6: a synthetic (non-PHI) instance's cleartext http egress PASSES build_check, so it must - # also come up at serve. Before the fix the HTTP cell fail-closed to prod-PHI when unstamped and - # wrongly degraded the lane at serve. +async def test_serve_allows_declared_cleartext_http_outbound(store: MessageStore) -> None: + # Finding 6: a cleartext http egress that PASSES build_check must also come up at serve. Before the + # fix the HTTP cell fail-closed to prod-PHI when unstamped and wrongly degraded the lane at serve. + # + # The lane used to qualify via the synthetic data label; since ADR 0153 it qualifies via the + # per-destination declaration instead. The finding under test — that the LIVE build reads the + # stamped posture rather than fail-closing — is unchanged, and the declaration is a sharper probe + # of it: it must survive the trip through build_outbound_connection into the live rebuild. reg = Registry() reg.add_outbound( - build_outbound_connection("OB_REST", Rest(url="http://partner.example.com/ingest")) + build_outbound_connection( + "OB_REST", + Rest(url="http://partner.example.com/ingest"), + cleartext_accepted=True, + cleartext_reason="legacy partner endpoint has no TLS", + ) ) - runner = RegistryRunner(reg, store, poll_interval=0.02, hop_posture=SYNTHETIC) + runner = RegistryRunner(reg, store, poll_interval=0.02, hop_posture=PROD_PHI) await runner.start() try: assert runner.connection_failed("OB_REST") is None # lane built + live, not refused @@ -82,6 +94,20 @@ async def test_serve_allows_synthetic_cleartext_http_outbound(store: MessageStor await runner.stop() +async def test_serve_degrades_synthetic_cleartext_http_outbound(store: MessageStore) -> None: + """ADR 0153 at serve: the synthetic label no longer keeps an undeclared cleartext lane alive.""" + reg = Registry() + reg.add_outbound( + build_outbound_connection("OB_REST", Rest(url="http://partner.example.com/ingest")) + ) + runner = RegistryRunner(reg, store, poll_interval=0.02, hop_posture=SYNTHETIC_ENFORCING) + await runner.start() + try: + assert "OB_REST" in runner.degraded_connections() + finally: + await runner.stop() + + async def test_serve_prod_phi_still_refuses_cleartext_http(store: MessageStore) -> None: # The prod-PHI HTTP refusal is preserved by the stamped posture (not loosened by Fix A). reg = Registry() @@ -108,7 +134,12 @@ async def test_reload_rebuild_stamps_posture_no_spurious_refusal(store: MessageS reg1 = Registry() reg1.add_outbound(build_outbound_connection("OB_TCP", Tcp(host="127.0.0.1", port=5000))) reg1.add_outbound( - build_outbound_connection("OB_REST", Rest(url="http://partner.example.com/ingest")) + build_outbound_connection( + "OB_REST", + Rest(url="http://partner.example.com/ingest"), + cleartext_accepted=True, + cleartext_reason="legacy partner endpoint has no TLS", + ) ) await runner.reload(reg1) # must not raise InsecureHopRefused assert runner.connection_failed("OB_REST") is None diff --git a/tests/test_hop_refusal_wiring.py b/tests/test_hop_refusal_wiring.py index fd3c99bf..09c54c84 100644 --- a/tests/test_hop_refusal_wiring.py +++ b/tests/test_hop_refusal_wiring.py @@ -85,12 +85,87 @@ def test_destination_blank_reason_rejected() -> None: def test_source_attestation_field() -> None: - s = Source(type=ConnectorType.REMOTEFILE, settings={}, tls_hop_attested=True) + s = Source( + type=ConnectorType.REMOTEFILE, + settings={}, + tls_hop_attested=True, + tls_hop_attested_reason="terminated at the mesh sidecar", + ) assert s.tls_hop_attested is True with pytest.raises(ValueError, match="without"): Source(type=ConnectorType.REMOTEFILE, settings={}, tls_hop_attested_reason="x") +def test_attestation_now_requires_a_reason() -> None: + """ADR 0153 decision 2, retro-fitted: the flag alone is no longer a valid attestation. + + An attestation asserting *this hop is secure by means the engine cannot see* is the one input that + can silently ALLOW an enforcing cleartext hop, so it is precisely the claim that must carry a + written justification when it is audited. Both models enforce it, and so does [logging]'s sibling.""" + with pytest.raises(ValueError, match="requires tls_hop_attested_reason"): + Source(type=ConnectorType.REMOTEFILE, settings={}, tls_hop_attested=True) + with pytest.raises(ValueError, match="requires tls_hop_attested_reason"): + Destination( + name="OB", type=ConnectorType.REST, settings={"url": "http://x"}, tls_hop_attested=True + ) + + +# --- ADR 0153 decision 2: the cleartext-acceptance pair, load-validated ------------------------ +def test_destination_cleartext_acceptance_defaults_off() -> None: + d = Destination(name="OB", type=ConnectorType.REST, settings={"url": "https://x"}) + assert d.cleartext_accepted is False + assert d.cleartext_reason is None + + +def test_destination_cleartext_acceptance_with_reason_ok() -> None: + d = Destination( + name="OB", + type=ConnectorType.REST, + settings={"url": "http://legacy.internal"}, + cleartext_accepted=True, + cleartext_reason="vendor firmware predates TLS; segment is not isolated", + ) + assert d.cleartext_accepted is True + assert d.cleartext_reason is not None + + +def test_destination_cleartext_acceptance_rejects_incoherent_pairs() -> None: + """All three fail-loud rules: flag without reason, blank reason, reason without flag.""" + with pytest.raises(ValueError, match="requires cleartext_reason"): + Destination( + name="OB", + type=ConnectorType.REST, + settings={"url": "http://x"}, + cleartext_accepted=True, + ) + with pytest.raises(ValueError, match="must be non-empty"): + Destination( + name="OB", + type=ConnectorType.REST, + settings={"url": "http://x"}, + cleartext_accepted=True, + cleartext_reason=" ", + ) + with pytest.raises(ValueError, match="without cleartext_accepted"): + Destination( + name="OB", + type=ConnectorType.REST, + settings={"url": "http://x"}, + cleartext_reason="orphaned reason", + ) + + +def test_cleartext_acceptance_is_destination_only() -> None: + """ADR 0153 decision 2 is explicit: putting the pair on Source would be a setting nothing consumes. + + Inbound binds are governed by a different mechanism (_inbound_insecure_bind_permitted plus the four + exposed-gates), which this ADR does not change. Pinned so a later "symmetry" refactor cannot add a + dead inbound knob that reads like a working control.""" + assert "cleartext_accepted" not in Source.model_fields + assert "cleartext_reason" not in Source.model_fields + assert "cleartext_accepted" in Destination.model_fields + + # --- decision 7: the [ai]->HopPosture mapping (is_phi from data_class; enforcing from [security]) -- _ENFORCE = SecurityEnforcement.ENFORCE _WARN = SecurityEnforcement.WARN @@ -181,3 +256,184 @@ def test_build_check_registry_none_posture_leaves_unstamped( ) # Unstamped -> None; the cell fail-closes on its own (treats the hop as prod-PHI). assert seen == [None] + + +# --- ADR 0153 decision 2: the AUTHORING surfaces for the cleartext-acceptance pair -------------- + + +def test_outbound_factory_accepts_the_pair_and_validates_it() -> None: + """`outbound()` / `build_outbound_connection` is the code-first surface. The coherence rules are + enforced at THIS choke point (not only on the model), so a bad pair fails at `messagefoundry check` + / dry-run with the connection name attached rather than at connector construction.""" + from messagefoundry.config.wiring import Tcp, WiringError + + oc = build_outbound_connection( + "OB", + Tcp(host="10.0.0.5", port=5000), + cleartext_accepted=True, + cleartext_reason="vendor firmware predates TLS", + ) + assert oc.cleartext_accepted is True and oc.cleartext_reason is not None + + with pytest.raises(WiringError, match="requires cleartext_reason"): + build_outbound_connection("OB", Tcp(host="10.0.0.5", port=5000), cleartext_accepted=True) + with pytest.raises(WiringError, match="without cleartext_accepted"): + build_outbound_connection("OB", Tcp(host="10.0.0.5", port=5000), cleartext_reason="why") + + +def test_connections_toml_desugars_to_the_same_declaration(tmp_path) -> None: # type: ignore[no-untyped-def] + """ADR 0007's promise: the TOML surface desugars through the SAME factories into an identical + Registry entry. Asserted on the DISPOSITION, not just the field, so a pair that survives the loader + but never reaches the hop authority would still fail here.""" + from messagefoundry.config.wiring import Tcp, load_config + from messagefoundry.pipeline.wiring_runner import _dest_config + + (tmp_path / "logic.py").write_text( + "from messagefoundry import Send, handler, router\n\n" + '@router("r")\n' + "def route(msg):\n" + ' return ["h"]\n\n' + '@handler("h")\n' + "def handle(msg):\n" + ' return Send("OB", msg)\n', + encoding="utf-8", + ) + (tmp_path / "connections.toml").write_text( + "[[outbound]]\n" + 'name = "OB"\n' + 'transport = "tcp"\n' + "cleartext_accepted = true\n" + 'cleartext_reason = "vendor firmware predates TLS"\n' + " [outbound.settings]\n" + ' host = "10.0.0.5"\n' + " port = 5000\n", + encoding="utf-8", + ) + from_toml = load_config(tmp_path).outbound["OB"] + from_code = build_outbound_connection( + "OB", + Tcp(host="10.0.0.5", port=5000), + cleartext_accepted=True, + cleartext_reason="vendor firmware predates TLS", + ) + assert (from_toml.cleartext_accepted, from_toml.cleartext_reason) == ( + from_code.cleartext_accepted, + from_code.cleartext_reason, + ) + # ...and both reach the typed Destination the connectors decide on. + for oc in (from_toml, from_code): + dest = _dest_config(oc, {}) + assert dest.cleartext_accepted is True + assert dest.cleartext_reason == "vendor firmware predates TLS" + + +def test_toml_rejects_the_pair_under_settings(tmp_path) -> None: # type: ignore[no-untyped-def] + """They are TOP-LEVEL outbound keys, not transport settings. Under `[settings]` they are passed to + the transport factory (which IS the settings schema) and rejected — pinned so the two surfaces + cannot silently diverge into accepting both spellings with different semantics. + + Driven through the LOADER, not through a factory signature: a signature check is a proxy that would + still pass if `_build_spec` ever grew a pass-through for unknown keys, which is the drift this test + is for.""" + from messagefoundry.config.wiring import WiringError, load_config + + (tmp_path / "logic.py").write_text( + "from messagefoundry import Send, handler, router\n\n" + '@router("r")\n' + "def route(msg):\n" + ' return ["h"]\n\n' + '@handler("h")\n' + "def handle(msg):\n" + ' return Send("OB", msg)\n', + encoding="utf-8", + ) + (tmp_path / "connections.toml").write_text( + "[[outbound]]\n" + 'name = "OB"\n' + 'transport = "tcp"\n' + " [outbound.settings]\n" + ' host = "10.0.0.5"\n' + " port = 5000\n" + " cleartext_accepted = true\n", + encoding="utf-8", + ) + with pytest.raises(WiringError, match="cleartext_accepted"): + load_config(tmp_path) + + +# --- ADR 0153 decision 3: NO TLS default is flipped -------------------------------------------- + + +def test_no_transport_factory_flips_its_tls_default() -> None: + """ADR 0153's most emphatic non-goal, pinned so it cannot rot. + + An earlier draft proposed `tls: bool = True` on the transport factories. It was sized and REJECTED: + it is redundant (an undeclared cleartext hop already refuses), it hard-fails every inbound MLLP + listener (`_mllp_ssl_context(server=True)` raises before any policy runs — including on the loopback + binds arm 1 exempts), and it disarms the four inbound exposed-gates, which early-return when + `settings["tls"]` is truthy. "Secure by default" is delivered by the REFUSAL, not by this default. + """ + import inspect + + from messagefoundry.config.wiring import DICOM, MLLP, Ftp, Http + + for factory in (MLLP, Http, DICOM, Ftp): + param = inspect.signature(factory).parameters["tls"] + assert param.default is False, ( + f"{factory.__name__}() now defaults tls={param.default!r}. ADR 0153 decision 3 keeps it " + "False on all four factories — see the ADR before changing this." + ) + + +def test_dest_config_mirrors_the_declaration_into_the_resolved_settings() -> None: + """The runner's settings MIRROR is the only path a graph-authored declaration takes to the deep + settings-driven seams — HTTP Digest, the OAuth2 / SMART token endpoints, the forward-proxy + credential. Nothing else covers it: those seams' own tests hand the settings dict in by hand, and + the desugaring test asserts only the typed `Destination` fields. Without this, that mirroring could + be deleted and a declared connection using `http_auth=digest` or `oauth2_*` over cleartext would die + at construction with no test noticing.""" + from messagefoundry.config.wiring import Tcp + from messagefoundry.pipeline.wiring_runner import _dest_config + + declared = build_outbound_connection( + "OB_LEGACY", + Tcp(host="10.0.0.5", port=5000), + cleartext_accepted=True, + cleartext_reason="vendor firmware predates TLS", + ) + settings = _dest_config(declared, {}).settings + assert settings["cleartext_accepted"] is True + assert settings["cleartext_reason"] == "vendor firmware predates TLS" + # The NAME rides with it, so the acceptance audit record those seams emit can name the declaration. + assert settings["cleartext_connection"] == "OB_LEGACY" + + +def test_dest_config_writes_no_mirror_keys_when_nothing_is_declared() -> None: + """The other half: an undeclared outbound must be byte-identical — no empty governance keys in the + resolved settings, which several surfaces render.""" + from messagefoundry.config.wiring import Tcp + from messagefoundry.pipeline.wiring_runner import _dest_config + + plain = build_outbound_connection("OB_PLAIN", Tcp(host="10.0.0.5", port=5000)) + settings = _dest_config(plain, {}).settings + assert "cleartext_accepted" not in settings + assert "cleartext_reason" not in settings + assert "cleartext_connection" not in settings + + +def test_logging_forward_hop_attestation_also_requires_a_reason() -> None: + """Rule 3 of the retro-fit reaches `[logging].forward_hop_attested` — the documented `[logging]` + sibling of a connection's `tls_hop_attested`, sharing the same validator. + + Asserted here rather than claimed in a sibling test's docstring: docs/PHI.md already described this + reason as mandatory, so an unenforced rule would leave a documented guarantee false exactly where an + auditor looks.""" + from messagefoundry.config.settings import LoggingSettings + + with pytest.raises(ValueError, match="requires tls_hop_attested_reason"): + LoggingSettings(forward_hop_attested=True) + # ...and the coherent pair still loads. + ok = LoggingSettings( + forward_hop_attested=True, forward_hop_attested_reason="management segment is isolated" + ) + assert ok.forward_hop_attested is True diff --git a/tests/test_http_auth.py b/tests/test_http_auth.py index cd896e92..e2cca430 100644 --- a/tests/test_http_auth.py +++ b/tests/test_http_auth.py @@ -155,13 +155,22 @@ def test_oauth2_cleartext_token_endpoint_refused_on_prod_phi_even_with_escape( assert "s3cr3t" not in str(ei.value) # never echo the secret in the refusal -def test_oauth2_cleartext_token_endpoint_allowed_non_prod_with_escape( +def test_oauth2_cleartext_token_endpoint_allowed_when_accepted( monkeypatch: pytest.MonkeyPatch, ) -> None: - # Non-prod (staging) PHI with the clamped escape set downgrades REFUSE→WARN — permitted (as-is). - monkeypatch.setenv(INSECURE_TLS_ESCAPE_ENV, "1") - with active_hop_posture(_STAGING_PHI): - p = _oauth_provider(token_url="http://auth.example.com/token") + """ADR 0153: the token-endpoint hop takes the same per-connection declaration as its delivery hop. + + It replaces the escape this test used to rely on (decision 5 unhooked that). Threading the pair here + is deliberate even though a credential is worse on the wire than a body: without it, an operator + whose legacy peer needs OAuth2 over a cleartext segment would have to write a FALSE + `tls_hop_attested`, which is the exact defect ADR 0153 exists to remove.""" + monkeypatch.delenv(INSECURE_TLS_ESCAPE_ENV, raising=False) + with active_hop_posture(_PROD_PHI): + p = _oauth_provider( + token_url="http://auth.example.com/token", + cleartext_accepted=True, + cleartext_reason="legacy IdP has no TLS listener", + ) assert isinstance(p, OAuth2ClientCredentialsProvider) @@ -188,11 +197,19 @@ def test_digest_cleartext_refused_on_prod_phi_even_with_escape( assert "cleartext" in str(ei.value) -def test_digest_cleartext_allowed_non_prod_with_escape(monkeypatch: pytest.MonkeyPatch) -> None: - monkeypatch.setenv(INSECURE_TLS_ESCAPE_ENV, "1") - with active_hop_posture(_STAGING_PHI): +def test_digest_cleartext_allowed_when_accepted(monkeypatch: pytest.MonkeyPatch) -> None: + # ADR 0153: the declaration (mirrored into the resolved settings by _dest_config) replaces the + # escape this test used to rely on. WARN + audit, not a silent crossing. + monkeypatch.delenv(INSECURE_TLS_ESCAPE_ENV, raising=False) + with active_hop_posture(_PROD_PHI): h = digest_handler_from_settings( - {"http_auth": "digest", "http_auth_user": "u", "http_auth_password": "p"}, + { + "http_auth": "digest", + "http_auth_user": "u", + "http_auth_password": "p", + "cleartext_accepted": True, + "cleartext_reason": "legacy device has no TLS listener", + }, url="http://api.example.com/x", ) assert h is not None # WARN, not REFUSE — the challenge-answering handler is still built diff --git a/tests/test_memory_encryption_readout.py b/tests/test_memory_encryption_readout.py index ea6f2066..f4dfc2ae 100644 --- a/tests/test_memory_encryption_readout.py +++ b/tests/test_memory_encryption_readout.py @@ -41,7 +41,23 @@ MemoryEncryptionReadout, platform_memory_encryption_readout, ) -from messagefoundry.config.settings import SecuritySettings, load_settings, security_loosenings +from messagefoundry.config.settings import ( + AuthSettings, + SecuritySettings, + StoreSettings, + load_settings, + security_loosenings, +) + + +def _loosenings(sec: SecuritySettings) -> list[tuple[str, str]]: + """``security_loosenings`` with the shipped [store]/[auth] defaults and an empty accepted set. + + The registry takes all four inputs as REQUIRED arguments deliberately (ADR 0148: one posture, and a + deviation the registry cannot see is a second posture by the back door). The tests below are about + the ``[security]`` switches specifically, so the other three are pinned at shipped values here.""" + return security_loosenings(sec, StoreSettings(), AuthSettings(), ()) + SAMPLES_CONFIG = Path(__file__).resolve().parents[1] / "samples" / "config" @@ -273,8 +289,8 @@ def test_setting_defaults_false_and_is_byte_identical_when_unset(tmp_path: Path) """Every shipped default is unchanged, and the switch is not a loosening (it asserts a protection rather than giving one up), so an all-defaults instance names nothing new anywhere.""" assert SecuritySettings().memory_encryption_operator_declared is False - assert security_loosenings(SecuritySettings()) == [] - assert security_loosenings(SecuritySettings(memory_encryption_operator_declared=True)) == [] + assert _loosenings(SecuritySettings()) == [] + assert _loosenings(SecuritySettings(memory_encryption_operator_declared=True)) == [] cfg = tmp_path / "messagefoundry.toml" cfg.write_text("", encoding="utf-8") diff --git a/tests/test_outbound_forward_proxy.py b/tests/test_outbound_forward_proxy.py index 14b70a49..f6755fca 100644 --- a/tests/test_outbound_forward_proxy.py +++ b/tests/test_outbound_forward_proxy.py @@ -54,7 +54,10 @@ def _rsa_pem() -> str: ).decode() -_SYNTHETIC = HopPosture(is_phi=False, enforcing=False) # no PHI on the wire → no dest-side refusals +# ADR 0153: the data label no longer relaxes a cleartext hop, so the permissive posture for these +# PROXY-behaviour tests is the non-enforcing dial (which still WARNs, never refuses). Renamed from +# _SYNTHETIC so nothing here reads as if the label were still doing the work. +_WARN_DIAL = HopPosture(is_phi=False, enforcing=False) _PROD = HopPosture( is_phi=True, enforcing=True ) # production PHI → the cleartext-proxy-hop guard bites @@ -76,10 +79,14 @@ def _build( ctype: ConnectorType, url: str, *, - posture: HopPosture = _SYNTHETIC, + posture: HopPosture = _WARN_DIAL, attested: bool = False, + accepted: bool = False, **over: object, ) -> object: + """Build one outbound. ``accepted`` is ADR 0153's per-connection cleartext declaration — needed by + the tests whose PROXY hop is plain http, since the proxy crossing is decided by the same authority + as the destination crossing.""" settings = _FACTORY[ctype](url=url, **over).settings # type: ignore[operator] with active_hop_posture(posture): return build_destination( @@ -88,6 +95,9 @@ def _build( type=ctype, settings=settings, tls_hop_attested=attested, + tls_hop_attested_reason="proxy-terminated trusted segment" if attested else None, + cleartext_accepted=accepted, + cleartext_reason="on-prem proxy listener has no TLS" if accepted else None, tls_revocation_attested=True, # isolate the proxy behaviour from the #201 revocation gate ) ) @@ -148,6 +158,9 @@ def test_basic_proxy_auth_preemptive_header() -> None: dest = _build( ConnectorType.REST, HTTPS_DEST, + # The PROXY hop is cleartext http and carries a credential, so under ADR 0153 it needs the + # connection's declaration to cross — the destination hop here is https and unaffected. + accepted=True, proxy=PROXY, proxy_user="pu", proxy_password="pw", @@ -300,11 +313,17 @@ def test_proxy_bypasses_unit() -> None: def test_token_endpoint_is_proxied() -> None: """AC-8: the OAuth2 token-endpoint call is routed through the connection's forward proxy too (opener - ProxyHandler + pre-emptive Proxy-Authorization).""" - with active_hop_posture(_SYNTHETIC): + ProxyHandler + pre-emptive Proxy-Authorization). + + The proxy hop here is cleartext http and carries a credential, so it needs the ADR 0153 declaration + to cross — the proxy-credential chain reads it off the resolved settings, the same mapping it + already reads ``tls_hop_attested`` from.""" + with active_hop_posture(_WARN_DIAL): cfg = proxy_config_from_settings( {"proxy_url": PROXY, "proxy_user": "pu", "proxy_password": "pw"}, dest_scheme="https", + cleartext_accepted=True, + cleartext_reason="on-prem proxy listener has no TLS", ) assert isinstance(cfg, ProxyConfig) provider = OAuth2ClientCredentialsProvider( @@ -321,10 +340,12 @@ def test_token_endpoint_is_proxied() -> None: def test_smart_token_endpoint_is_proxied() -> None: """AC-8 (SMART leg): the SMART Backend Services token-endpoint POST is proxied too — its OWN opener carries the ProxyHandler and its own pre-emptive Proxy-Authorization (distinct wiring from OAuth2).""" - with active_hop_posture(_SYNTHETIC): + with active_hop_posture(_WARN_DIAL): cfg = proxy_config_from_settings( {"proxy_url": PROXY, "proxy_user": "pu", "proxy_password": "pw"}, dest_scheme="https", + cleartext_accepted=True, + cleartext_reason="on-prem proxy listener has no TLS", ) assert isinstance(cfg, ProxyConfig) provider = SmartBackendTokenProvider( @@ -361,7 +382,7 @@ def test_egress_default_proxy() -> None: def test_fhir_lookup_executor_proxied() -> None: """The fhir_lookup read hop (and its SMART token endpoint) traverse the proxy too.""" - with active_hop_posture(_SYNTHETIC): + with active_hop_posture(_WARN_DIAL): ex = FhirLookupExecutor( { "epic": { diff --git a/tests/test_phi_at_rest_inventory.py b/tests/test_phi_at_rest_inventory.py index c8d09eed..14157b6d 100644 --- a/tests/test_phi_at_rest_inventory.py +++ b/tests/test_phi_at_rest_inventory.py @@ -466,7 +466,7 @@ def test_cited_settings_names_resolve() -> None: def test_documented_defaults_match_the_shipped_defaults() -> None: """A default flip must force the §3/§8 wording to be revisited, not rot silently.""" settings = ServiceSettings() - assert settings.store.aad_bind is False, "§3 says cell binding is OFF by default" + assert settings.store.aad_bind is True, "§3 says cell binding is ON by default" assert settings.store.cipher_provider == "aesgcm", ( "§3 says the default cipher is in-process AES-GCM" ) diff --git a/tests/test_postgres_store.py b/tests/test_postgres_store.py index 77bb8c22..f43a2b79 100644 --- a/tests/test_postgres_store.py +++ b/tests/test_postgres_store.py @@ -3569,6 +3569,7 @@ async def test_rotate_key_cli_reencrypts_server_store(store, capsys, monkeypatch (retired key dropped) — end-to-end proof.""" from messagefoundry.__main__ import main from messagefoundry.config.settings import load_settings + from messagefoundry.store.crypto import AesGcmCipher from messagefoundry.store.postgres import PostgresStore monkeypatch.chdir( @@ -3576,7 +3577,11 @@ async def test_rotate_key_cli_reencrypts_server_store(store, capsys, monkeypatch ) # isolate from any stray ./messagefoundry.toml (env wins, but be safe) settings = load_settings(environ=os.environ).store key_a, key_b = generate_key(), generate_key() - active_id_b = make_cipher(key_b).active_key_id + # The rotation writer is NOT an object this test holds: main(["rotate-key"]) -> open_store -> + # build_store_cipher(settings), i.e. write_v2=[store].aad_bind. Mirror that posture so the marker + # this test expects is the one the CLI actually wrote, whichever way the aad_bind default sits. + cipher_b = make_cipher(key_b, write_v2=settings.aad_bind) + assert isinstance(cipher_b, AesGcmCipher) seed = await PostgresStore.open(settings, cipher=make_cipher(key_a)) try: @@ -3590,9 +3595,7 @@ async def test_rotate_key_cli_reencrypts_server_store(store, capsys, monkeypatch assert rc == 0 assert "re-encrypted" in capsys.readouterr().out - verify = await PostgresStore.open( - settings, cipher=make_cipher(key_b) - ) # key_b alone, no retired + verify = await PostgresStore.open(settings, cipher=cipher_b) # key_b alone, no retired try: assert len(await verify.list_messages()) == 1 assert (await verify.get_message(mid))["raw"] == RAW # decrypts under the new key alone @@ -3602,7 +3605,7 @@ async def test_rotate_key_cli_reencrypts_server_store(store, capsys, monkeypatch assert blobs # at least messages.raw + the one outbound payload for r in blobs: assert r["v"].startswith(MARKER_PREFIX) - assert r["v"].split(":", 3)[2] == active_id_b # mfenc:v1:: + assert r["v"].startswith(cipher_b.active_marker_prefix) # active key, active format finally: await verify.close() diff --git a/tests/test_rest_transport.py b/tests/test_rest_transport.py index cc909e1e..7314c22e 100644 --- a/tests/test_rest_transport.py +++ b/tests/test_rest_transport.py @@ -26,10 +26,24 @@ URL = "https://api.example.com/ingest" -def _dest(**over: object) -> RestDestination: - """Build a RestDestination from Rest(...) settings (env() refs already 'resolved' = literals).""" +def _dest( + *, _cleartext_accepted: bool = False, _cleartext_reason: str | None = None, **over: object +) -> RestDestination: + """Build a RestDestination from Rest(...) settings (env() refs already 'resolved' = literals). + + ``_cleartext_accepted``/``_cleartext_reason`` (ADR 0153) are underscore-prefixed because they are + NOT transport settings — they are top-level outbound keys threaded onto the Destination, and mixing + them into ``**over`` would send them to the ``Rest(...)`` factory, which rightly rejects them.""" settings = Rest(url=URL, **over).settings # type: ignore[arg-type] - d = build_destination(Destination(name="OB_REST", type=ConnectorType.REST, settings=settings)) + d = build_destination( + Destination( + name="OB_REST", + type=ConnectorType.REST, + settings=settings, + cleartext_accepted=_cleartext_accepted, + cleartext_reason=_cleartext_reason, + ) + ) assert isinstance(d, RestDestination) return d @@ -148,14 +162,38 @@ def test_rest_verify_tls_false_refused_without_escape(monkeypatch: pytest.Monkey def test_rest_verify_tls_false_allowed_with_escape(monkeypatch: pytest.MonkeyPatch) -> None: + """A verify-off hop keeps the CLAMPED global escape — ADR 0153 does not govern this cell. + + 0153 unhooked ``MEFOR_ALLOW_INSECURE_TLS`` from the CLEARTEXT hop decision. A ``verify_tls=false`` + hop is encrypted-but-unauthenticated, not cleartext, so it is explicitly out of that scope and + decides exactly as the MLLP/FTPS ``tls_verify=false`` cells do.""" monkeypatch.setenv("MEFOR_ALLOW_INSECURE_TLS", "1") - # #200 (ADR 0092): the global escape now only DOWNGRADES REFUSE→WARN on a NON-production instance - # (decision 2). Under a non-prod PHI posture it warns-and-builds; on production it would refuse. with active_hop_posture(HopPosture(is_phi=True, enforcing=False)): dest = _dest(verify_tls=False) # builds a no-verify opener; no exception assert dest._opener is not None +def test_rest_verify_tls_false_not_relaxed_by_a_cleartext_declaration( + monkeypatch: pytest.MonkeyPatch, +) -> None: + """``cleartext_accepted`` must NOT cross a verify-off hop, in any posture. + + Threading it here would have been a LOOSENING — this hop REFUSES on an enforcing instance today, + and ADR 0092 decision 5 forbids a cell getting weaker. It would also have attached an operator's + written "this peer cannot do TLS" reason to a peer that plainly does TLS, and split the HTTP family + from MLLP, which decides the same question through ``weakened_tls_escape_permitted_here()``.""" + monkeypatch.delenv("MEFOR_ALLOW_INSECURE_TLS", raising=False) + with ( + active_hop_posture(HopPosture(is_phi=True, enforcing=True)), + pytest.raises(ValueError, match="verify_tls=false"), + ): + _dest( + verify_tls=False, + _cleartext_accepted=True, + _cleartext_reason="legacy partner endpoint has no TLS", + ) + + def test_rest_credentials_over_cleartext_http_refused(monkeypatch: pytest.MonkeyPatch) -> None: # Basic/bearer over plain http leaks the credential — refused unless the explicit escape is set. monkeypatch.delenv("MEFOR_ALLOW_INSECURE_TLS", raising=False) @@ -169,20 +207,24 @@ def test_rest_credentials_over_cleartext_http_refused(monkeypatch: pytest.Monkey ) -def test_rest_credentials_over_cleartext_http_allowed_with_escape( +def test_rest_credentials_over_cleartext_http_allowed_when_accepted( monkeypatch: pytest.MonkeyPatch, ) -> None: - monkeypatch.setenv("MEFOR_ALLOW_INSECURE_TLS", "1") - # #200: the escape downgrades REFUSE→WARN only on a NON-production instance (decision 2). - with active_hop_posture(HopPosture(is_phi=True, enforcing=False)): + # ADR 0153: the blunt MEFOR_ALLOW_INSECURE_TLS escape no longer influences a cleartext-hop + # decision (decision 5). The per-connection declaration is what crosses it now — loudly, and + # recorded in the audit trail, instead of a process-wide env var nobody sees in review. + monkeypatch.delenv("MEFOR_ALLOW_INSECURE_TLS", raising=False) + with active_hop_posture(HopPosture(is_phi=True, enforcing=True)): dest = build_destination( Destination( name="OB", type=ConnectorType.REST, settings=Rest(url="http://api.example.com/x", bearer_token="tok").settings, + cleartext_accepted=True, + cleartext_reason="legacy partner endpoint has no TLS", ) ) - assert isinstance(dest, RestDestination) # built (warns), not refused + assert isinstance(dest, RestDestination) # built (warns + audits), not refused def test_rest_cleartext_http_without_credentials_is_allowed() -> None: @@ -224,20 +266,24 @@ def test_rest_cleartext_http_nonloopback_refused_without_escape( ) -def test_rest_cleartext_http_nonloopback_allowed_with_escape( +def test_rest_cleartext_http_nonloopback_allowed_when_accepted( monkeypatch: pytest.MonkeyPatch, ) -> None: - monkeypatch.setenv("MEFOR_ALLOW_INSECURE_TLS", "1") - # #200: the escape downgrades REFUSE→WARN only on a NON-production instance (decision 2). - with active_hop_posture(HopPosture(is_phi=True, enforcing=False)): + # ADR 0153: the blunt MEFOR_ALLOW_INSECURE_TLS escape no longer influences a cleartext-hop + # decision (decision 5). The per-connection declaration is what crosses it now — loudly, and + # recorded in the audit trail, instead of a process-wide env var nobody sees in review. + monkeypatch.delenv("MEFOR_ALLOW_INSECURE_TLS", raising=False) + with active_hop_posture(HopPosture(is_phi=True, enforcing=True)): dest = build_destination( Destination( name="OB", type=ConnectorType.REST, settings=Rest(url="http://api.example.com/x").settings, + cleartext_accepted=True, + cleartext_reason="legacy partner endpoint has no TLS", ) ) - assert isinstance(dest, RestDestination) # built (warns loudly), not refused + assert isinstance(dest, RestDestination) # built (warns loudly + audits), not refused def test_rest_egress_allowlist_blocks_unlisted_host() -> None: diff --git a/tests/test_security_cli.py b/tests/test_security_cli.py index 49e183d1..2124a697 100644 --- a/tests/test_security_cli.py +++ b/tests/test_security_cli.py @@ -125,3 +125,45 @@ def test_set_rejects_the_allowlist_beside_a_broad_trusted_proxies( rc, _ = _set(toml, {"allowed_client_networks": ["10.20.0.0/16"]}, capsys) assert rc == 1 assert "[security]" not in toml.read_text(encoding="utf-8") # rolled back + + +def test_show_declares_that_it_cannot_see_connection_scoped_deviations( + tmp_path: Path, capsys: pytest.CaptureFixture[str] +) -> None: + """`security show` reads a SETTINGS file and never loads the connection graph, so it cannot see the + ADR 0153 per-connection ``cleartext_accepted`` declarations. + + The marker is the whole mitigation: without it this subcommand reports a settings-only list that + READS as the complete posture, which under "one posture, loosen only" is exactly how a deviation + stops being visible. Nothing else pins it, so it could otherwise be dropped silently.""" + toml = tmp_path / "mf.toml" + data = _show(toml, capsys) + assert data["loosenings_partial"] is False + assert "cleartext_accepted" in data["loosenings_scope"] + assert "messagefoundry check" in data["loosenings_scope"] + + +def test_show_reports_store_and_auth_deviations_from_the_whole_file( + tmp_path: Path, capsys: pytest.CaptureFixture[str] +) -> None: + """The registry reaches into [store]/[auth], so this subcommand must resolve the WHOLE file rather + than [security] alone — otherwise it under-reports the posture it exists to display.""" + toml = tmp_path / "mf.toml" + toml.write_text("[store]\naad_bind = false\n", encoding="utf-8") + data = _show(toml, capsys) + assert "aad_bind" in [entry["switch"] for entry in data["loosenings"]] + + +def test_show_declares_a_partial_report_when_the_file_will_not_load( + tmp_path: Path, capsys: pytest.CaptureFixture[str] +) -> None: + """A file invalid OUTSIDE [security] must not break `security show` — but the degraded report has to + SAY it is degraded, not quietly fall back to shipped defaults and look complete.""" + toml = tmp_path / "mf.toml" + # Valid TOML, invalid SETTINGS, and invalid entirely outside [security]: an explicit + # ad_session_recheck_seconds with no ad_enabled is the ADR 0079 cross-field refusal. + toml.write_text("[auth]\nad_session_recheck_seconds = 300\n", encoding="utf-8") + data = _show(toml, capsys) + assert data["loosenings_partial"] is True + # ...and it still prints a usable [security] view rather than failing the subcommand. + assert data["values"]["require_mfa"] is True diff --git a/tests/test_security_config.py b/tests/test_security_config.py index 32e7eeba..d2e8cc21 100644 --- a/tests/test_security_config.py +++ b/tests/test_security_config.py @@ -18,12 +18,24 @@ from messagefoundry.__main__ import main from messagefoundry.config.ai_policy import DataClass from messagefoundry.config.settings import ( + AuthSettings, SecuritySettings, ServiceSettings, + StoreSettings, load_settings, security_loosenings, ) + +def _loosenings(sec: SecuritySettings) -> list[tuple[str, str]]: + """``security_loosenings`` with the shipped [store]/[auth] defaults and an empty accepted set. + + The registry takes all four inputs as REQUIRED arguments deliberately (ADR 0148: one posture, and a + deviation the registry cannot see is a second posture by the back door). The tests below are about + the ``[security]`` switches specifically, so the other three are pinned at shipped values here.""" + return security_loosenings(sec, StoreSettings(), AuthSettings(), ()) + + SAMPLES_CONFIG = Path(__file__).resolve().parents[1] / "samples" / "config" @@ -194,14 +206,12 @@ def test_loosening_warns_and_prod_phi_refuses( ) -> None: # security_loosenings() names each opt-out in plain language (the serve warning + the posture view # both consume it) — audit_all_authorization_decisions=false is NOT a loosening (secure default). - loos = dict( - security_loosenings(SecuritySettings(require_mfa=False, block_unlisted_outbound=False)) - ) + loos = dict(_loosenings(SecuritySettings(require_mfa=False, block_unlisted_outbound=False))) assert "require_mfa" in loos and "single-factor" in loos["require_mfa"] assert ( "block_unlisted_outbound" in loos and "any destination" in loos["block_unlisted_outbound"] ) - assert security_loosenings(SecuritySettings()) == [] # all-secure defaults → nothing named + assert _loosenings(SecuritySettings()) == [] # all-secure defaults → nothing named # The serve-time consolidated warning fires naming the loosened switch (AC-4). It rides the logging # path (post-configure_logging), which routes to stdout — the gate REFUSE messages print to stderr. @@ -243,7 +253,7 @@ def test_production_acks_are_loosenings_when_set() -> None: # allow_keeping_phi. Each appears exactly once (guards against a duplicate-append bug). switches = [ k - for k, _ in security_loosenings( + for k, _ in _loosenings( SecuritySettings( allow_single_factor_admin_when_exposed=True, allow_unencrypted_phi_under_strict_enforcement=True, @@ -254,7 +264,7 @@ def test_production_acks_are_loosenings_when_set() -> None: # is genuinely verified, per ADR 0140 AC-6. assert switches.count("allow_single_factor_admin_when_exposed") == 1 assert switches.count("allow_unencrypted_phi_under_strict_enforcement") == 1 - assert security_loosenings(SecuritySettings()) == [] # acks off => nothing named + assert _loosenings(SecuritySettings()) == [] # acks off => nothing named # --- AC-6: handles_real_patient_data=false relaxes the PHI-only gates (and it is posture-visible) -- @@ -314,12 +324,10 @@ def test_enforcement_default_and_env_override(tmp_path: Path) -> None: def test_enforcement_warn_is_named_as_a_loosening_once() -> None: from messagefoundry.config.ai_policy import SecurityEnforcement - switches = [ - k for k, _ in security_loosenings(SecuritySettings(enforcement=SecurityEnforcement.WARN)) - ] + switches = [k for k, _ in _loosenings(SecuritySettings(enforcement=SecurityEnforcement.WARN))] assert switches.count("enforcement") == 1 # ENFORCE (the secure default) is NOT a loosening. - assert "enforcement" not in dict(security_loosenings(SecuritySettings())) + assert "enforcement" not in dict(_loosenings(SecuritySettings())) def test_enforcement_decouples_refuse_warn_from_tier( diff --git a/tests/test_security_doc_drift.py b/tests/test_security_doc_drift.py index 4be2fbca..f12de451 100644 --- a/tests/test_security_doc_drift.py +++ b/tests/test_security_doc_drift.py @@ -367,7 +367,7 @@ # These three were documented but unpinned — precisely the defaults the trailing lanes plan to # move (#297's 8.3.2 route proposes an ADR-0080-style derived ad_session_recheck_seconds), so a # change would have made the row false with zero CI signal. - ("auth", "ad_session_recheck_seconds", 0, "**0 = off**"), + ("auth", "ad_session_recheck_seconds", 300, "**300 s**"), ("auth", "phi_read_rate_limit_global", 0, "`0` = **off**"), ("auth", "phi_read_rate_limit_window_seconds", 60.0, "60 s"), ("auth", "oidc_flow_cache_max", 512, "**512**"), diff --git a/tests/test_security_posture_defaults.py b/tests/test_security_posture_defaults.py new file mode 100644 index 00000000..89931f52 --- /dev/null +++ b/tests/test_security_posture_defaults.py @@ -0,0 +1,502 @@ +# SPDX-License-Identifier: AGPL-3.0-or-later +# Copyright (C) 2026 MessageFoundry Organization and contributors +"""The shipped-default posture lane: ONE posture, and operators may only LOOSEN from it. + +Two defaults moved to the hardened value (ADR 0148 GIVEN 1 — the hardened path is the shipped path, so it +is the path every test, CI leg and dogfood instance exercises, not one first met in production): + +* ``[store].aad_bind`` ``false`` → **``true``** (ADR 0019, 2026-07-28 amendment) — at-rest values are + cell-bound (``mfenc:v2``) by default; +* ``[auth].ad_session_recheck_seconds`` ``0`` → **``300``** (ADR 0079, 2026-07-28 amendment) — directory + revocation propagates by default. + +The governing rule is that every deviation from that one posture is VISIBLE: ``security_loosenings()`` + +``GET /security/posture`` + ``docs/SECURITY-LOOSENING.md``. A deviation the registry cannot see is a +second posture by the back door, so these tests are as much about the REGISTRY as about the defaults — +including a completeness floor, because a registry with no floor is exactly the shape that lets a later +switch be added at an insecure value with nothing reporting it. +""" + +from __future__ import annotations + +from pathlib import Path + +import httpx +import pytest + +from messagefoundry.api import create_app +from messagefoundry.config.settings import ( + AuthSettings, + SecuritySettings, + ServiceSettings, + StoreSettings, + load_settings, + security_loosenings, +) +from messagefoundry.pipeline import Engine + + +def _ad(**over: object) -> AuthSettings: + """AD-enabled auth settings with the connection essentials the model requires.""" + base: dict[str, object] = { + "ad_enabled": True, + "ad_server": "ldaps://dc.test.invalid", + "ad_user_search_base": "OU=Staff,DC=test,DC=invalid", + "ad_bind_dn": "CN=svc-mefor,OU=Service,DC=test,DC=invalid", + "ad_bind_password": "synthetic", + } + base.update(over) + return AuthSettings(**base) # type: ignore[arg-type] + + +def _names( + sec: SecuritySettings | None = None, + store: StoreSettings | None = None, + auth: AuthSettings | None = None, + cleartext_hops: tuple[str, ...] = (), +) -> list[str]: + """The loosening SWITCH NAMES for a settings combination (defaults where not overridden).""" + return [ + name + for name, _ in security_loosenings( + sec or SecuritySettings(), + store or StoreSettings(), + auth or AuthSettings(), + cleartext_hops, + ) + ] + + +# --- the shipped defaults themselves --------------------------------------------------------- + + +def test_shipped_defaults_are_the_hardened_values() -> None: + """Both flips, pinned at the model. A default that moves back reds here first.""" + settings = ServiceSettings() + assert settings.store.aad_bind is True + assert settings.auth.ad_session_recheck_seconds == 300 + + +def test_the_shipped_defaults_are_not_themselves_loosenings() -> None: + """The whole point: at the shipped defaults the registry reports NOTHING. If a hardened default + were reported as a deviation, the list would be noise and operators would stop reading it.""" + assert _names() == [] + + +# --- [store].aad_bind ------------------------------------------------------------------------ + + +def test_aad_bind_off_is_a_named_loosening() -> None: + named = dict( + security_loosenings(SecuritySettings(), StoreSettings(aad_bind=False), AuthSettings(), ()) + ) + assert "aad_bind" in named + # The risk text must say what is actually lost — cell binding, i.e. at-rest INTEGRITY binding — not + # merely that a switch is off. An operator reading the serve warning gets this sentence and nothing + # else; "aad_bind is false" would tell them nothing they did not already know. + assert "cell" in named["aad_bind"] + + +def test_aad_bind_loosening_names_its_no_op_caveat() -> None: + """It is a genuine no-op without a store key (the identity cipher has no tag to bind), and the risk + text says so. Reporting it as a live weakness on a keyless dev box would train operators to ignore + the list — the failure mode a loosening registry can least afford.""" + named = dict( + security_loosenings(SecuritySettings(), StoreSettings(aad_bind=False), AuthSettings(), ()) + ) + assert "no effect without a store key" in named["aad_bind"] + + +# --- [auth].ad_session_recheck_seconds ------------------------------------------------------- + + +def test_recheck_zero_with_ad_enabled_is_a_named_loosening() -> None: + auth = _ad(ad_session_recheck_seconds=0) + named = dict(security_loosenings(SecuritySettings(), StoreSettings(), auth, ())) + assert "ad_session_recheck_seconds" in named + assert "revocation" in named["ad_session_recheck_seconds"] + + +def test_recheck_zero_without_ad_is_NOT_a_loosening() -> None: + """CONDITIONAL, like allowed_client_networks. With no directory to reconcile against, 0 is not a + weaker choice — it is the only meaningful one. + + This is the detector-can-fire half of the guard: a rule that fired on every non-AD deployment would + be a permanent false positive, and a permanently-true warning is read as noise, not as signal.""" + assert "ad_session_recheck_seconds" not in _names( + auth=AuthSettings(ad_session_recheck_seconds=0) + ) + + +def test_recheck_at_the_default_with_ad_enabled_is_not_a_loosening() -> None: + assert "ad_session_recheck_seconds" not in _names(auth=_ad()) + + +# --- the cross-field refusal, keyed on model_fields_set --------------------------------------- +# +# These go through load_settings, NOT the constructor. Constructing AuthSettings(...) in Python marks +# every passed field as "set", so a constructor-only test cannot distinguish the shipped default from an +# explicitly-typed 300 — which is the entire distinction the guard turns on. It would pass while proving +# nothing. + + +def _load(tmp_path: Path, toml: str) -> ServiceSettings: + path = tmp_path / "messagefoundry.toml" + path.write_text(toml, encoding="utf-8") + return load_settings(config_path=path) + + +def test_shipped_default_does_not_break_a_non_ad_deployment(tmp_path: Path) -> None: + """The reason the refusal had to be re-keyed: with a non-zero SHIPPED default, an unconditional + 'requires ad_enabled' rule would fail startup on every deployment that does not use AD.""" + settings = _load(tmp_path, "[auth]\nlocal_users = true\n") + assert settings.auth.ad_session_recheck_seconds == 300 + assert settings.auth.ad_enabled is False + + +def test_explicit_value_without_ad_still_refuses(tmp_path: Path) -> None: + """THE test that proves the guard still bites. An operator who typed a value believes directory + revocation now propagates; a silently-dead security control is worse than one never enabled. + + Note the value is the SAME as the shipped default — so this can only pass if the check keys on + `model_fields_set` rather than on the value.""" + with pytest.raises(ValueError, match="ad_session_recheck_seconds requires ad_enabled"): + _load(tmp_path, "[auth]\nad_session_recheck_seconds = 300\n") + + +def test_explicit_zero_without_ad_loads(tmp_path: Path) -> None: + """Explicitly disabling the loop on a non-AD box is coherent, not an error — there is nothing to + reconcile, and the operator has asserted no belief the refusal needs to falsify.""" + settings = _load(tmp_path, "[auth]\nad_session_recheck_seconds = 0\n") + assert settings.auth.ad_session_recheck_seconds == 0 + + +# --- registry completeness -------------------------------------------------------------------- + + +def test_every_security_bool_at_its_insecure_value_is_reported() -> None: + """A COMPLETENESS FLOOR for the registry, which otherwise has none. + + Nothing else asserts that `security_loosenings()` can SEE every switch. Under "one posture, loosen + only", a registry with no floor is the leak-gate-blindness shape: a switch added later at an insecure + value would simply never be reported, and the green list would keep saying "no deviations". + + Scope, stated honestly: this covers the BOOLEAN `[security]` switches whose insecure value is the + negation of their default — the mechanical majority. Non-boolean knobs (timeouts, day counts) and the + deliberately-conditional entries have their own targeted tests above and below; they are listed here + as exemptions so the exemption itself is visible rather than an accident of the loop.""" + #: Bools this floor deliberately does NOT require, each with the reason it is exempt. + exempt = { + # Owner-confirmed secure-and-usable default; turning it ON is the hardening move, not the + # loosening (ADR 0118 §5) — it is documented as "not a loosening" in SECURITY-LOOSENING.md. + "audit_all_authorization_decisions", + # ADR 0152: these ASSERT / REQUIRE a host property rather than giving one up. Neither is a + # loosening at either value; both are documented as such. + "memory_encryption_operator_declared", + "require_memory_encryption_declaration", + # ADR 0143: disabling the console SHRINKS attack surface — the opposite of a loosening. + "serve_web_console", + # The data-class lever has its own entry keyed on the derived posture, not a plain negation. + "handles_real_patient_data", + "production_instance", + } + for field, info in SecuritySettings.model_fields.items(): + if field in exempt or not isinstance(info.default, bool): + continue + flipped = SecuritySettings(**{field: not info.default}) + assert field in _names(sec=flipped), ( + f"[security].{field} at its insecure value ({not info.default}) is NOT named by " + "security_loosenings(). Add it to the registry, or add it to this test's `exempt` set " + "with the reason it is not a loosening — silence is not an option." + ) + + +# --- the API surface: GET /security/posture reports store + auth deviations -------------------- + + +async def _posture_body(engine: Engine, **state: object) -> dict[str, object]: + app = create_app(engine, allow_no_auth=True) + for key, value in state.items(): + setattr(app.state, key, value) + transport = httpx.ASGITransport(app=app) + async with httpx.AsyncClient(transport=transport, base_url="http://t") as client: + resp = await client.get("/security/posture") + assert resp.status_code == 200 + body: dict[str, object] = resp.json() + return body + + +@pytest.fixture +async def engine(tmp_path: Path): # type: ignore[no-untyped-def] + eng = await Engine.create(tmp_path / "posture.db", poll_interval=0.02) + yield eng + await eng.stop() + + +async def test_posture_route_reports_the_store_deviation(engine: Engine) -> None: + body = await _posture_body(engine, store_settings=StoreSettings(aad_bind=False)) + switches = [entry["switch"] for entry in body["loosenings"]] # type: ignore[index,union-attr] + assert "aad_bind" in switches + + +async def test_posture_route_reports_the_auth_deviation(engine: Engine) -> None: + body = await _posture_body(engine, auth_settings=_ad(ad_session_recheck_seconds=0)) + switches = [entry["switch"] for entry in body["loosenings"]] # type: ignore[index,union-attr] + assert "ad_session_recheck_seconds" in switches + + +async def test_posture_route_reports_nothing_at_the_shipped_defaults(engine: Engine) -> None: + """The route must be quiet on a default instance, or its signal is worthless.""" + body = await _posture_body(engine) + assert body["loosenings"] == [] + + +# --- the ONE connection-scoped deviation (ADR 0153) -------------------------------------------- + + +def test_cleartext_accepted_is_a_named_loosening() -> None: + """ADR 0153's per-connection declaration MUST surface in the same registry as the settings + switches. It is a deviation from the one shipped posture, and a deviation the registry cannot see is + a second posture by the back door.""" + named = dict( + security_loosenings( + SecuritySettings(), StoreSettings(), AuthSettings(), ("OB_LEGACY", "OB_LAB") + ) + ) + assert "cleartext_accepted" in named + risk = named["cleartext_accepted"] + # It must NAME the connections. "some connections cross a cleartext hop" is not actionable — an + # operator has to know WHICH, because the remedy is per-connection. + assert "OB_LEGACY" in risk and "OB_LAB" in risk + assert "2 connection(s)" in risk + + +def test_no_declared_hops_is_not_a_loosening() -> None: + assert "cleartext_accepted" not in _names(cleartext_hops=()) + + +def test_accepted_cleartext_hops_reads_the_graph() -> None: + """The single shared reader — `messagefoundry check` and the API posture route both use it, so the + two can never report different accepted sets.""" + from messagefoundry.config.models import ConnectorType + from messagefoundry.config.wiring import ( + ConnectionSpec, + Registry, + accepted_cleartext_hops, + build_outbound_connection, + ) + + reg = Registry() + reg.add_outbound( + build_outbound_connection( + "OB_PLAIN", ConnectionSpec(type=ConnectorType.TCP, settings={"host": "x", "port": 1}) + ) + ) + reg.add_outbound( + build_outbound_connection( + "OB_LEGACY", + ConnectionSpec(type=ConnectorType.TCP, settings={"host": "y", "port": 2}), + cleartext_accepted=True, + cleartext_reason="vendor firmware predates TLS", + ) + ) + assert accepted_cleartext_hops(reg) == [("OB_LEGACY", "vendor firmware predates TLS")] + + +async def test_posture_route_reports_declared_cleartext_hops(engine: Engine) -> None: + """The surface the owner named explicitly. The route reads the LIVE graph off the engine's registry + runner, so a reload is reflected rather than a startup snapshot going stale.""" + from messagefoundry.config.models import ConnectorType + from messagefoundry.config.wiring import ConnectionSpec, Registry, build_outbound_connection + + reg = Registry() + reg.add_outbound( + build_outbound_connection( + "OB_LEGACY", + ConnectionSpec(type=ConnectorType.TCP, settings={"host": "127.0.0.1", "port": 5099}), + cleartext_accepted=True, + cleartext_reason="vendor firmware predates TLS", + ) + ) + engine.add_registry(reg) + body = await _posture_body(engine) + entry = next( + e + for e in body["loosenings"] # type: ignore[union-attr] + if e["switch"] == "cleartext_accepted" # type: ignore[index] + ) + assert "OB_LEGACY" in entry["risk"] # type: ignore[index] + + +def test_declared_fhir_lookup_read_hops_are_named_too() -> None: + """A ``FhirLookup`` is a connection that crosses a PHI-bearing read hop, and the read executor + honours the declaration — so if this reader skipped ``registry.fhir_lookups`` a live cleartext hop + would cross while `check`, `security_loosenings()` and `GET /security/posture` all reported the + accepted set as EMPTY. That is precisely "a deviation the registry cannot see".""" + from messagefoundry.config import wiring + from messagefoundry.config.wiring import FhirLookup, Registry, accepted_cleartext_hops + + reg = Registry() + prev = wiring._active + wiring._active = reg + try: + FhirLookup("quiet", url="https://fhir.example.org/fhir") + FhirLookup( + "legacy", + url="http://fhir.example.org/fhir", + cleartext_accepted=True, + cleartext_reason="on-prem facade has no TLS listener", + ) + finally: + wiring._active = prev + assert accepted_cleartext_hops(reg) == [ + ("fhir_lookup:legacy", "on-prem facade has no TLS listener") + ] + + +def test_fhir_lookup_declaration_is_load_validated() -> None: + """The factory is the ONE authoring surface, so the flag/reason coherence rule must fire there. + + Before this, the only way to declare it on a lookup was mutating ``spec.settings`` by hand — an + escape with no validation and nothing for the registry to name.""" + from messagefoundry.config import wiring + from messagefoundry.config.wiring import FhirLookup, Registry, WiringError + + reg = Registry() + prev = wiring._active + wiring._active = reg + try: + with pytest.raises(WiringError, match="requires cleartext_reason"): + FhirLookup("x", url="http://f.example.org/fhir", cleartext_accepted=True) + with pytest.raises(WiringError, match="without cleartext_accepted"): + FhirLookup("y", url="http://f.example.org/fhir", cleartext_reason="why") + with pytest.raises(WiringError, match="must be non-empty"): + FhirLookup( + "z", + url="http://f.example.org/fhir", + cleartext_accepted=True, + cleartext_reason=" ", + ) + finally: + wiring._active = prev + + +def test_every_store_and_auth_bool_is_reported_or_exempt() -> None: + """The completeness floor, extended over the two OTHER sections this registry reaches into. + + The floor above covers ``[security]`` only. Without this one, the registry's reach into + ``[store]``/``[auth]`` would be exactly the leak-gate-blindness shape one section over: a green + "no deviations" that has never looked. The exemption set below is the honest part — it enumerates + the switches that are NOT reported today, so the gap is a written decision rather than an + accident, and a NEW switch in either section cannot silently join it.""" + #: Not reported by security_loosenings() today. Each is gated elsewhere; extending the registry + #: over them is real work with its own SECURITY-LOOSENING.md entries, and is recorded as owed + #: rather than done silently here. A new field in either section reds this test until it is + #: either reported or added here with a reason. + exempt_store = { + # Not security switches at all — FIFO-claim performance levers and pool knobs. + "fifo_claim_fold_reset", + "fifo_claim_proc", + "fifo_claim_prepared", + "multi_subnet_failover", + "warm_pool", + # HARDENINGS at their non-default value (turning them ON tightens), so a flip is not a loosening. + "require_encryption", + "require_managed_identity", + # Security-relevant and gated ELSEWHERE, not by this registry. Extending it over them is real + # work with its own SECURITY-LOOSENING.md entries — recorded as owed, not done silently. + "encrypt", # the keyless-PHI serve gate refuses it in its own right + "trust_server_certificate", # gated by weakened_tls_escape_permitted (the ADR 0092 clamp) + "allow_unencrypted_phi", # reported via [security].allow_unencrypted_phi (ADR 0118 move) + } + exempt_auth = { + # HARDENINGS / topology choices — a flip is not a weakening of the shipped posture. + "require_action_step_up", + "admin_new_ip_step_up", + "ad_enabled", + "ad_use_nested_groups", + "kerberos_enabled", + "oidc_enabled", + "oidc_username_strip_domain", + "notify_security_events", + # Password-policy composition rules: individually neither secure nor insecure (the policy is + # scored as a whole), and none is a posture switch. + "password_require_uppercase", + "password_require_lowercase", + "password_require_digit", + "password_require_symbol", + "password_check_context", + "password_check_username", + "password_check_breached", + # Security-relevant and gated ELSEWHERE, not by this registry — same owed note as [store]. + "enabled", # the serve-time exposed-gates refuse an exposed auth-off instance outright + "require_mfa", # refused at exposure by the __main__ posture gates + "ad_tls_verify", # gated by weakened_tls_escape_permitted + "ad_allow_insecure_ldap", # gated by the same clamp + "oidc_require_mfa_claim", # gated by the OIDC serve gate + "login_rate_limit_enabled", # DoS hardening with its own serve-time defaults + "phi_read_rate_limit_enabled", + "admin_write_rate_limit_enabled", + } + for model, exempt, section in ( + (StoreSettings, exempt_store, "store"), + (AuthSettings, exempt_auth, "auth"), + ): + for field, info in model.model_fields.items(): + if field in exempt or not isinstance(info.default, bool): + continue + flipped = model(**{field: not info.default}) # type: ignore[arg-type] + kwargs = {"store": flipped} if section == "store" else {"auth": flipped} + assert field in _names(**kwargs), ( # type: ignore[arg-type] + f"[{section}].{field} at its insecure value ({not info.default}) is NOT named by " + "security_loosenings(). Add it to the registry, or add it to this test's exemption " + "set with the reason — silence is not an option." + ) + + +async def test_posture_route_declares_its_scope_when_no_graph_is_loaded(engine: Engine) -> None: + """An engine with no registry runner cannot see the connection-scoped declarations, so it SAYS so. + + Reporting a settings-only list with no marker is the failure this whole lane exists to prevent: + a subset that reads as the whole posture. `security show` carries the same marker for the same + reason, and this pins that the route does not quietly differ from it.""" + body = await _posture_body(engine) + assert body["loosenings_scope"] is not None + assert "cleartext_accepted" in str(body["loosenings_scope"]) + + +async def test_posture_route_scope_is_none_once_a_graph_is_loaded(engine: Engine) -> None: + """The complementary arm — the marker must CLEAR, or it degrades into permanent noise.""" + from messagefoundry.config.wiring import Registry + + engine.add_registry(Registry()) + body = await _posture_body(engine) + assert body["loosenings_scope"] is None + + +async def test_managed_app_stashes_auth_settings_for_the_registry(tmp_path: Path) -> None: + """Drive the REAL wiring: `create_managed_app` must stash `auth_settings` on app.state, or the + route silently falls back to `AuthSettings()` defaults and the auth deviation is never reported. + + The targeted route test above sets `app.state.auth_settings` by hand, so it cannot fail if the + stash regresses. This one goes through the lifespan, which is the only thing that proves the + production path is wired.""" + from messagefoundry.api import create_managed_app + + app = create_managed_app( + db_path=tmp_path / "managed_posture.db", + poll_interval=0.05, + # enabled=False so the route stays reachable without a session; the stash is deliberately + # OUTSIDE the `enabled` guard, and that is exactly what this pins — a settings object that + # exists but is disabled is still the resolved settings the registry must read. + auth_settings=_ad(ad_session_recheck_seconds=0, enabled=False), + ) + transport = httpx.ASGITransport(app=app) + async with ( + httpx.AsyncClient(transport=transport, base_url="http://t") as client, + app.router.lifespan_context(app), + ): + resp = await client.get("/security/posture") + assert resp.status_code == 200 + switches = [entry["switch"] for entry in resp.json()["loosenings"]] + assert "ad_session_recheck_seconds" in switches diff --git a/tests/test_smart_backend.py b/tests/test_smart_backend.py index 1efcb7de..6108ecaa 100644 --- a/tests/test_smart_backend.py +++ b/tests/test_smart_backend.py @@ -314,6 +314,69 @@ def test_cleartext_token_url_refused(rsa_pem: str) -> None: ) +def test_cleartext_token_url_is_decided_by_the_one_authority( + monkeypatch: pytest.MonkeyPatch, rsa_pem: str +) -> None: + """The SMART token endpoint used to read the RAW, UNCLAMPED ``MEFOR_ALLOW_INSECURE_TLS``. + + One process-wide environment variable therefore put a signed ``client_assertion`` on cleartext http + even on an enforcing PHI instance — the exact "variable alive here, dead there" asymmetry ADR 0153 + decision 5 removes everywhere else, and it was invisible to the loosening registry. This cell now + goes through ``refuse_cleartext_credential_hop`` like its OAuth2 sibling, so the blunt escape cannot + cross it.""" + from messagefoundry.config.tls_policy import HopPosture, active_hop_posture + + monkeypatch.setenv("MEFOR_ALLOW_INSECURE_TLS", "1") + with ( + active_hop_posture(HopPosture(is_phi=True, enforcing=True)), + pytest.raises(SmartAuthError, match="cleartext"), + ): + SmartBackendTokenProvider( + token_url="http://auth.example/token", client_id="c", private_key=rsa_pem + ) + + +def test_cleartext_token_url_crosses_on_a_per_connection_declaration( + monkeypatch: pytest.MonkeyPatch, rsa_pem: str +) -> None: + """The other arm: the connection's OWN declaration crosses it — loudly, audited, and named by the + loosening registry. Without it, an operator whose legacy IdP has no TLS listener would be pushed + into writing a false ``tls_hop_attested``, the defect ADR 0153 exists to remove.""" + from messagefoundry.config.tls_policy import HopPosture, active_hop_posture + + monkeypatch.delenv("MEFOR_ALLOW_INSECURE_TLS", raising=False) + with active_hop_posture(HopPosture(is_phi=True, enforcing=True)): + provider = SmartBackendTokenProvider( + token_url="http://auth.example/token", + client_id="c", + private_key=rsa_pem, + cleartext_accepted=True, + cleartext_reason="legacy IdP has no TLS listener", + connection="OB_LEGACY", + ) + assert provider.token_url == "http://auth.example/token" + + +def test_token_provider_from_settings_reads_the_declaration(rsa_pem: str) -> None: + """The settings-driven seam: `_dest_config` mirrors the declaration (and the connection name) into + the resolved settings, and this factory must READ it — otherwise the graph-authored pair never + reaches the token-endpoint gate and a declared connection dies at construction.""" + from messagefoundry.config.tls_policy import HopPosture, active_hop_posture + from messagefoundry.transports.smart import token_provider_from_settings + + settings = { + "smart_token_url": "http://auth.example/token", + "smart_client_id": "c", + "smart_private_key": rsa_pem, + "cleartext_accepted": True, + "cleartext_reason": "legacy IdP has no TLS listener", + "cleartext_connection": "OB_LEGACY", + } + with active_hop_posture(HopPosture(is_phi=True, enforcing=True)): + provider = token_provider_from_settings(settings) + assert provider is not None + + def test_missing_client_id_refused(rsa_pem: str) -> None: with pytest.raises(SmartAuthError, match="client_id"): SmartBackendTokenProvider(token_url=TOKEN_URL, client_id="", private_key=rsa_pem) diff --git a/tests/test_soap_transport.py b/tests/test_soap_transport.py index 4e411409..f5ae3b98 100644 --- a/tests/test_soap_transport.py +++ b/tests/test_soap_transport.py @@ -245,20 +245,24 @@ def test_soap_cleartext_http_loopback_allowed() -> None: assert isinstance(dest, SoapDestination) -def test_soap_cleartext_http_nonloopback_allowed_with_escape( +def test_soap_cleartext_http_nonloopback_allowed_when_accepted( monkeypatch: pytest.MonkeyPatch, ) -> None: - monkeypatch.setenv("MEFOR_ALLOW_INSECURE_TLS", "1") - # #200 (ADR 0092): the escape downgrades REFUSE→WARN only on a NON-production instance (decision 2). - with active_hop_posture(HopPosture(is_phi=True, enforcing=False)): + # ADR 0153: the blunt MEFOR_ALLOW_INSECURE_TLS escape no longer influences a cleartext-hop + # decision (decision 5). The per-connection declaration is what crosses it now — loudly, and + # recorded in the audit trail, instead of a process-wide env var nobody sees in review. + monkeypatch.delenv("MEFOR_ALLOW_INSECURE_TLS", raising=False) + with active_hop_posture(HopPosture(is_phi=True, enforcing=True)): dest = build_destination( Destination( name="OB", type=ConnectorType.SOAP, settings=Soap(url="http://api.example.com/svc").settings, + cleartext_accepted=True, + cleartext_reason="legacy partner endpoint has no TLS", ) ) - assert isinstance(dest, SoapDestination) # built (warns loudly), not refused + assert isinstance(dest, SoapDestination) # built (warns loudly + audits), not refused def test_egress_shares_allowed_http() -> None: diff --git a/tests/test_sqlserver_store.py b/tests/test_sqlserver_store.py index ce917cbe..fe85a5d1 100644 --- a/tests/test_sqlserver_store.py +++ b/tests/test_sqlserver_store.py @@ -1012,8 +1012,11 @@ async def test_reencrypt_to_active_rotates_all_columns_including_state(store) -> finally: await old.close() - active_id = AesGcmCipher(k2).active_key_id - rotated = await SqlServerStore.open(settings, cipher=AesGcmCipher(k2, retired_keys=[k1])) + # Assert against the very cipher that does the rewriting, via the marker prefix IT writes — so the + # check pins marker + version + key id together (and the alg too, on v2) and stays right whatever + # at-rest format is active. The `== 6` below is the independent proof that rotation really ran. + active_cipher = AesGcmCipher(k2, retired_keys=[k1]) + rotated = await SqlServerStore.open(settings, cipher=active_cipher) try: # messages.raw + the outbound queue.payload + state.value (3 core) + the 3 lifecycle-event # message_events.detail rows this flow writes — all now ciphered under H4 — = 6 under the @@ -1027,7 +1030,7 @@ async def test_reencrypt_to_active_rotates_all_columns_including_state(store) -> " UNION ALL SELECT payload FROM queue WHERE payload <> ''" ) for r in blobs: - assert r["v"].split(":", 3)[2] == active_id, r["v"] # mfenc:v1:: + assert r["v"].startswith(active_cipher.active_marker_prefix), r["v"] assert dict(rotated.state_view())[("ns", "k")] == {"v": 1} # still decrypts assert await rotated.reencrypt_to_active() == 0 # idempotent finally: @@ -1352,7 +1355,7 @@ async def test_reingress_depth_cap_dead_letters(store) -> None: async def test_response_rotation_and_purge(store) -> None: from messagefoundry.config.settings import load_settings - from messagefoundry.store.crypto import AesGcmCipher + from messagefoundry.store.crypto import AesGcmCipher, _fingerprint from messagefoundry.store.sqlserver import SqlServerStore settings = load_settings(environ=os.environ).store @@ -1366,14 +1369,22 @@ async def test_response_rotation_and_purge(store) -> None: finally: await old.close() - active_id = AesGcmCipher(k2).active_key_id - rotated = await SqlServerStore.open(settings, cipher=AesGcmCipher(k2, retired_keys=[k1])) + # Assert through the rotating cipher's own marker prefix, not a hand-rolled split of one field — + # the shipped accessor tracks whatever at-rest format the writer emits (v1 pins marker+version+key + # id; v2 pins the alg too). Paired with the retired fingerprint's ABSENCE, which is derived from a + # DIFFERENT property (_fingerprint, not active_marker_prefix) so the two still cross-check: were + # the prefix ever to go over-broad, rotation would silently no-op and startswith pass vacuously. + retired_id = _fingerprint(k1) + active_cipher = AesGcmCipher(k2, retired_keys=[k1]) + rotated = await SqlServerStore.open(settings, cipher=active_cipher) try: await rotated.reencrypt_to_active() # rotates response.body + detail (among others) blobs = await rotated._fetchall("SELECT body, detail FROM response") + assert blobs, "expected a rotated response row" for b in blobs: - assert b["body"].split(":", 3)[2] == active_id # mfenc:v1::... - assert b["detail"].split(":", 3)[2] == active_id + for v in (b["body"], b["detail"]): + assert v.startswith(active_cipher.active_marker_prefix), v + assert retired_id not in v r = (await rotated.correlate_response(mid))[0] assert ( r.body == "secret-reply" and r.detail == "secret-detail" @@ -1425,7 +1436,7 @@ async def test_reingress_peek_failed_errors_child_and_skips_ingress(store) -> No async def test_reencrypt_skips_null_response_detail(store) -> None: from messagefoundry.config.settings import load_settings - from messagefoundry.store.crypto import AesGcmCipher + from messagefoundry.store.crypto import AesGcmCipher, _fingerprint from messagefoundry.store.sqlserver import SqlServerStore settings = load_settings(environ=os.environ).store @@ -1438,13 +1449,18 @@ async def test_reencrypt_skips_null_response_detail(store) -> None: finally: await old.close() - active_id = AesGcmCipher(k2).active_key_id - rotated = await SqlServerStore.open(settings, cipher=AesGcmCipher(k2, retired_keys=[k1])) + retired_id = _fingerprint(k1) + active_cipher = AesGcmCipher(k2, retired_keys=[k1]) + rotated = await SqlServerStore.open(settings, cipher=active_cipher) try: await rotated.reencrypt_to_active() # must not crash on / mis-handle the NULL detail row = (await rotated._fetchall("SELECT body, detail FROM response"))[0] assert row["detail"] is None # NULL skipped (IS NOT NULL guard), not crashed - assert row["body"].split(":", 3)[2] == active_id # body rotated to the active key + # The row itself was NOT skipped: body carries the rotating cipher's active marker prefix, + # and no longer the retired fingerprint (checked off _fingerprint, so it cross-checks the + # prefix rather than restating it — an over-broad prefix would make startswith vacuous). + assert row["body"].startswith(active_cipher.active_marker_prefix) + assert retired_id not in row["body"] r = (await rotated.correlate_response(mid))[0] assert r.body == "reply-body" and r.detail is None finally: @@ -1486,7 +1502,7 @@ async def test_reencrypt_rotates_summary_and_metadata(store) -> None: """EF-3: summary/metadata are rotated under the active key like the body — not stranded under a retired key (which a later retired-key drop would make undecryptable).""" from messagefoundry.config.settings import load_settings - from messagefoundry.store.crypto import AesGcmCipher + from messagefoundry.store.crypto import AesGcmCipher, _fingerprint from messagefoundry.store.sqlserver import SqlServerStore settings = load_settings(environ=os.environ).store @@ -1500,13 +1516,18 @@ async def test_reencrypt_rotates_summary_and_metadata(store) -> None: finally: await old.close() - active_id = AesGcmCipher(k2).active_key_id - rotated = await SqlServerStore.open(settings, cipher=AesGcmCipher(k2, retired_keys=[k1])) + retired_id = _fingerprint(k1) + active_cipher = AesGcmCipher(k2, retired_keys=[k1]) + rotated = await SqlServerStore.open(settings, cipher=active_cipher) try: await rotated.reencrypt_to_active() row = (await rotated._fetchall("SELECT summary, metadata FROM messages"))[0] - assert row["summary"].split(":", 3)[2] == active_id # mfenc:v1:: - assert row["metadata"].split(":", 3)[2] == active_id + # Under the ACTIVE key in the ACTIVE marker format — the prefix the rotating cipher writes — + # and no longer under the retired one. The second check comes off _fingerprint, not + # active_marker_prefix, so it independently proves rotation ran rather than restating it. + for v in (row["summary"], row["metadata"]): + assert v.startswith(active_cipher.active_marker_prefix), v + assert retired_id not in v [m] = await rotated.list_messages() assert ( m["summary"] == summary and m["metadata"] == metadata @@ -1517,8 +1538,10 @@ async def test_reencrypt_rotates_summary_and_metadata(store) -> None: # --- H4: error / last_error / message_events.detail encrypted at rest ---------- # SQL Server parity with SQLite/Postgres: the three nullable disposition-text columns route through the -# SAME store cipher (mfenc:v1) — at-rest ciphertext, decrypt-on-read, rotated on rekey, and legacy -# plaintext migrated on open. The prior "SQL Server keeps these plaintext" residual is retired. +# SAME store cipher — at-rest ciphertext in whichever mfenc format that cipher writes, decrypt-on-read, +# rotated on rekey, and legacy plaintext migrated on open. (Naming a version here would be wrong: the +# format follows [store].aad_bind, so it is v2 by default.) The prior "SQL Server keeps these +# plaintext" residual is retired. async def test_error_lasterror_detail_encrypted_at_rest_and_decrypt(store) -> None: @@ -1572,7 +1595,7 @@ async def test_reencrypt_rotates_error_lasterror_detail(store) -> None: """H4: the three disposition-text columns are rotated to the active key on rekey like the body — not stranded under a retired key (which a later retired-key drop would make undecryptable).""" from messagefoundry.config.settings import load_settings - from messagefoundry.store.crypto import AesGcmCipher + from messagefoundry.store.crypto import AesGcmCipher, _fingerprint from messagefoundry.store.sqlserver import SqlServerStore settings = load_settings(environ=os.environ).store @@ -1587,11 +1610,15 @@ async def test_reencrypt_rotates_error_lasterror_detail(store) -> None: finally: await old.close() - active_id = AesGcmCipher(k2).active_key_id - rotated = await SqlServerStore.open(settings, cipher=AesGcmCipher(k2, retired_keys=[k1])) + retired_id = _fingerprint(k1) + active_cipher = AesGcmCipher(k2, retired_keys=[k1]) + rotated = await SqlServerStore.open(settings, cipher=active_cipher) try: await rotated.reencrypt_to_active() - # every non-null disposition-text value now carries the ACTIVE key id (mfenc:v1::...). + # every non-null disposition-text value now carries the ACTIVE key id, in whichever marker + # format the rotating cipher writes (active_marker_prefix spans marker + version + fingerprint, + # plus the alg on v2) — and none still carries the RETIRED fingerprint. That second check is + # derived from _fingerprint rather than the prefix, so it cross-checks rotation actually ran. blobs = await rotated._fetchall( "SELECT error AS v FROM messages WHERE error IS NOT NULL" " UNION ALL SELECT last_error FROM queue WHERE last_error IS NOT NULL" @@ -1599,7 +1626,8 @@ async def test_reencrypt_rotates_error_lasterror_detail(store) -> None: ) assert blobs, "expected rotated disposition-text values" for r in blobs: - assert r["v"].split(":", 3)[2] == active_id, r["v"] + assert r["v"].startswith(active_cipher.active_marker_prefix), r["v"] + assert retired_id not in r["v"], r["v"] # still decrypts under the new key on the read paths. assert any(m["error"] == err for m in await rotated.list_messages()) assert (await rotated.list_dead())[0]["last_error"] == fail @@ -2034,14 +2062,15 @@ async def test_reference_rows_rotate_on_reencrypt_to_active(store) -> None: finally: await old.close() - active_id = AesGcmCipher(k2).active_key_id - rotated = await SqlServerStore.open(settings, cipher=AesGcmCipher(k2, retired_keys=[k1])) + active_cipher = AesGcmCipher(k2, retired_keys=[k1]) + rotated = await SqlServerStore.open(settings, cipher=active_cipher) try: assert await rotated.reencrypt_to_active() == 2 # exactly the two reference rows rows = await rotated._fetchall("SELECT value FROM reference") assert len(rows) == 2 for r in rows: - assert r["value"].split(":", 3)[2] == active_id # mfenc:v1:: + # under the ACTIVE key, in the ACTIVE marker format; no plaintext PHI visible + assert r["value"].startswith(active_cipher.active_marker_prefix) assert "M-SECRET-1" not in r["value"] assert rotated.reference_view()["codes"]["P1"] == {"mrn": "M-SECRET-1"} # decrypts assert await rotated.reencrypt_to_active() == 0 # idempotent @@ -3563,6 +3592,7 @@ async def test_rotate_key_cli_reencrypts_server_store(store, capsys, monkeypatch and the message decrypts under key_b ALONE (retired key dropped) — end-to-end proof.""" from messagefoundry.__main__ import main from messagefoundry.config.settings import load_settings + from messagefoundry.store.crypto import AesGcmCipher from messagefoundry.store.sqlserver import SqlServerStore monkeypatch.chdir( @@ -3570,7 +3600,11 @@ async def test_rotate_key_cli_reencrypts_server_store(store, capsys, monkeypatch ) # isolate from any stray ./messagefoundry.toml (env wins, but be safe) settings = load_settings(environ=os.environ).store key_a, key_b = generate_key(), generate_key() - active_id_b = make_cipher(key_b).active_key_id + # The rotation writer is NOT an object this test holds: main(["rotate-key"]) -> open_store -> + # build_store_cipher(settings), i.e. write_v2=[store].aad_bind. Mirror that posture so the marker + # this test expects is the one the CLI actually wrote, whichever way the aad_bind default sits. + cipher_b = make_cipher(key_b, write_v2=settings.aad_bind) + assert isinstance(cipher_b, AesGcmCipher) seed = await SqlServerStore.open(settings, cipher=make_cipher(key_a)) try: @@ -3584,9 +3618,7 @@ async def test_rotate_key_cli_reencrypts_server_store(store, capsys, monkeypatch assert rc == 0 assert "re-encrypted" in capsys.readouterr().out - verify = await SqlServerStore.open( - settings, cipher=make_cipher(key_b) - ) # key_b alone, no retired + verify = await SqlServerStore.open(settings, cipher=cipher_b) # key_b alone, no retired try: assert len(await verify.list_messages()) == 1 assert (await verify.get_message(mid))["raw"] == RAW # decrypts under the new key alone @@ -3596,7 +3628,7 @@ async def test_rotate_key_cli_reencrypts_server_store(store, capsys, monkeypatch assert blobs # at least messages.raw + the one outbound payload for r in blobs: assert r["v"].startswith(MARKER_PREFIX) - assert r["v"].split(":", 3)[2] == active_id_b # mfenc:v1:: + assert r["v"].startswith(cipher_b.active_marker_prefix) # active key, active format finally: await verify.close() diff --git a/tests/test_store_encryption.py b/tests/test_store_encryption.py index ed275ce4..00241346 100644 --- a/tests/test_store_encryption.py +++ b/tests/test_store_encryption.py @@ -563,7 +563,8 @@ def test_marker_prefix_is_version_agnostic() -> None: # The v2 writer folds cell_aad(table, column, *pk) into the AES-GCM tag, so a ciphertext is bound to its # (table, column, row) cell: a blob cut-and-pasted into another cell fails the tag (CipherError) instead # of decrypting. v1 NEVER carries AAD (frozen, CRYPTO-1); decrypt dispatches AAD by marker version so -# legacy v1 rows still read (dual-read). Opt-in via [store].aad_bind → write_v2 (off by default). +# legacy v1 rows still read (dual-read). Selected by [store].aad_bind → write_v2 (ON by default +# since ADR 0148 GIVEN 1; the tests below pass an explicit cipher, so they pin the format directly). def test_cell_aad_unambiguous_and_backend_agnostic() -> None: diff --git a/tests/test_tls_policy.py b/tests/test_tls_policy.py index 1d706825..457998c1 100644 --- a/tests/test_tls_policy.py +++ b/tests/test_tls_policy.py @@ -4,6 +4,7 @@ from __future__ import annotations +import inspect import ssl import types from itertools import product @@ -227,70 +228,139 @@ def test_is_loopback_hop_host(host: str, expected: bool) -> None: assert is_loopback_hop_host(host) is expected +def _legacy_insecure_hop_disposition( + *, + is_phi: bool, + enforcing: bool, + is_loopback_hop: bool, + hop_attested: bool, + audited_opt_out: bool, +) -> HopDisposition: + """The PRE-ADR-0153 precedence, kept verbatim as the reference for the no-loosen property test. + + This is the only place the deleted ``not is_phi -> ALLOW`` arm still exists. Comparing the shipped + authority against it across the whole OLD input space is what turns ADR 0153's central claim — + "strictly stricter, provably" — into a check that can FAIL, rather than a sentence in a document. + """ + if is_loopback_hop: + return HopDisposition.ALLOW + if hop_attested: + return HopDisposition.ALLOW + if not is_phi: + return HopDisposition.ALLOW + if audited_opt_out: + return HopDisposition.WARN + if enforcing: + return HopDisposition.REFUSE + return HopDisposition.WARN + + def test_insecure_hop_disposition_full_precedence_table() -> None: - """Exhaustively assert the owner-ratified precedence over every input combination.""" - for is_phi, production, is_loopback_hop, hop_attested, audited_opt_out in product( - [False, True], repeat=5 + """Exhaustively assert the ADR 0153 precedence over every input combination (2**4, not 2**5).""" + for enforcing, is_loopback_hop, hop_attested, cleartext_accepted in product( + [False, True], repeat=4 ): got = insecure_hop_disposition( - is_phi=is_phi, - enforcing=production, + enforcing=enforcing, is_loopback_hop=is_loopback_hop, hop_attested=hop_attested, - audited_opt_out=audited_opt_out, + cleartext_accepted=cleartext_accepted, ) - # Explicit early-return precedence: loopback -> attested -> synthetic -> opt-out -> prod -> else. - if is_loopback_hop or hop_attested or not is_phi: + # Explicit early-return precedence: loopback -> attested -> accepted -> not-enforcing -> REFUSE. + if is_loopback_hop or hop_attested: expected = HopDisposition.ALLOW - elif audited_opt_out: + elif cleartext_accepted or not enforcing: expected = HopDisposition.WARN - elif production: - expected = HopDisposition.REFUSE else: - expected = HopDisposition.WARN - assert got is expected, (is_phi, production, is_loopback_hop, hop_attested, audited_opt_out) + expected = HopDisposition.REFUSE + assert got is expected, (enforcing, is_loopback_hop, hop_attested, cleartext_accepted) -def test_insecure_hop_prod_phi_refuses_and_escape_cannot_relax() -> None: - """The headline case: a prod-PHI hop refuses, and the (clamped-to-False on prod) escape cannot help.""" - base = dict(is_phi=True, enforcing=True, is_loopback_hop=False, hop_attested=False) # noqa: C408 - # The escape (audited_opt_out) is clamped to False on prod upstream (settings.hop_insecure_escape_ - # downgrades), so the realistic prod input is False here → REFUSE. - assert insecure_hop_disposition(**base, audited_opt_out=False) is HopDisposition.REFUSE - # Attestation is the ONLY per-hop way across a prod-PHI hop: - assert ( - insecure_hop_disposition( - is_phi=True, - enforcing=True, - is_loopback_hop=False, - hop_attested=True, - audited_opt_out=False, +def test_insecure_hop_disposition_no_longer_takes_a_data_label() -> None: + """ADR 0153 decisions 1 + 5, as a detector that CAN fail if either parameter is reintroduced. + + Asserting the arms alone would not catch a re-added ``is_phi=False -> ALLOW`` arm hidden behind a + defaulted parameter, because every existing call site would keep passing. The signature IS the + contract, so the signature is what is pinned.""" + params = set(inspect.signature(insecure_hop_disposition).parameters) + assert "is_phi" not in params, ( + "no data label may reach the cleartext-hop authority (ADR 0153 #1)" + ) + assert "audited_opt_out" not in params, ( + "MEFOR_ALLOW_INSECURE_TLS may no longer influence a cleartext-hop decision (ADR 0153 #5)" + ) + assert params == {"enforcing", "is_loopback_hop", "hop_attested", "cleartext_accepted"} + + +def test_insecure_hop_disposition_is_strictly_stricter_than_before() -> None: + """ADR 0153's central claim: over the OLD input space, no input moves toward ALLOW. + + "Because the deleted arm returned ALLOW, removing it can only ever turn a crossing into a WARN or a + REFUSE, never the reverse" — i.e. ADR 0092 decision 5 (no-loosen) holds by construction. This is the + executable form of that sentence: for every one of the old 32 rows the new disposition is never + *weaker* than the old one. ``cleartext_accepted`` is held FALSE throughout, since it is the new + deliberate loosening and would correctly relax a REFUSE to WARN.""" + strictness = {HopDisposition.ALLOW: 0, HopDisposition.WARN: 1, HopDisposition.REFUSE: 2} + for is_phi, enforcing, is_loopback_hop, hop_attested, audited_opt_out in product( + [False, True], repeat=5 + ): + before = _legacy_insecure_hop_disposition( + is_phi=is_phi, + enforcing=enforcing, + is_loopback_hop=is_loopback_hop, + hop_attested=hop_attested, + audited_opt_out=audited_opt_out, + ) + after = insecure_hop_disposition( + enforcing=enforcing, + is_loopback_hop=is_loopback_hop, + hop_attested=hop_attested, + cleartext_accepted=False, + ) + assert strictness[after] >= strictness[before], ( + is_phi, + enforcing, + is_loopback_hop, + hop_attested, + audited_opt_out, + before, + after, ) + + +def test_insecure_hop_enforcing_refuses_and_only_attestation_or_acceptance_crosses() -> None: + """The headline case: an enforcing cleartext hop refuses, whatever the instance's data label.""" + base = dict(enforcing=True, is_loopback_hop=False) # noqa: C408 + assert ( + insecure_hop_disposition(**base, hop_attested=False, cleartext_accepted=False) + is HopDisposition.REFUSE + ) + # Attestation ("this hop IS secure by other means") crosses it silently... + assert ( + insecure_hop_disposition(**base, hop_attested=True, cleartext_accepted=False) is HopDisposition.ALLOW ) + # ...and acceptance ("this hop is NOT secure and we accept that") crosses it LOUDLY, never as an + # ALLOW. That difference is the whole reason the two stay separate fields (ADR 0153 decision 2). + assert ( + insecure_hop_disposition(**base, hop_attested=False, cleartext_accepted=True) + is HopDisposition.WARN + ) -def test_insecure_hop_staging_phi_still_refuses_only_when_prod() -> None: - # Non-prod PHI (staging/dev-with-phi) WARNs, it does not refuse — the gradient adds coverage, it - # never turns a staging hop that only warns today into a refusal. +def test_insecure_hop_non_enforcing_warns_and_acceptance_still_only_warns() -> None: + # The [security].enforcement dial of ADR 0148 is deliberately RETAINED as arm 4: a non-enforcing + # instance warns rather than refuses, and still logs + audits every hop. Nothing goes silent. assert ( insecure_hop_disposition( - is_phi=True, - enforcing=False, - is_loopback_hop=False, - hop_attested=False, - audited_opt_out=False, + enforcing=False, is_loopback_hop=False, hop_attested=False, cleartext_accepted=False ) is HopDisposition.WARN ) - # ...and a non-prod audited opt-out also WARNs (the escape relaxes REFUSE->WARN only, non-prod). + # An acceptance on a non-enforcing instance is still WARN — it can never escalate to ALLOW. assert ( insecure_hop_disposition( - is_phi=True, - enforcing=False, - is_loopback_hop=False, - hop_attested=False, - audited_opt_out=True, + enforcing=False, is_loopback_hop=False, hop_attested=False, cleartext_accepted=True ) is HopDisposition.WARN ) diff --git a/tests/test_transform_state.py b/tests/test_transform_state.py index 7bdb0c54..991611d6 100644 --- a/tests/test_transform_state.py +++ b/tests/test_transform_state.py @@ -316,16 +316,20 @@ async def test_key_rotation_reencrypts_state_and_reads_still_work(tmp_path: Path await store.close() # Reopen with a NEW active key, old kept as retired (decrypt-only) — the rotation scenario. new = generate_key() - store2 = await MessageStore.open(db, cipher=make_cipher(new, [old])) + rotating = make_cipher(new, [old]) + store2 = await MessageStore.open(db, cipher=rotating) try: # Before rotation the value is still under the old key but reads work (keyring). assert store2.state_view()[("ns", "k")] == "secret" rotated = await store2.reencrypt_to_active() assert rotated >= 1 # the state value (among others) re-encrypted - # On disk it is now under the NEW key id, and reads still resolve. - new_id = make_cipher(new).active_key_id # type: ignore[attr-defined] + # On disk it is now under the NEW key id, and reads still resolve. Take the expected marker + # from the rotating cipher itself instead of hand-building f"{PREFIX}{new_id}:" — that spelling + # bakes in the v1 field order and misreads a v2 value (mfenc:v2:::), which is what + # [store].aad_bind makes the default at rest. `old_id not in` keeps the independent proof that + # rotation actually moved the value, derived from active_key_id rather than the prefix. at_rest = _state_at_rest(db, "ns", "k") - assert at_rest.startswith(f"{PREFIX}{new_id}:") + assert at_rest.startswith(rotating.active_marker_prefix) # type: ignore[attr-defined] assert old_id not in at_rest assert store2.state_view()[("ns", "k")] == "secret" finally: