Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 7 additions & 3 deletions .github/SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ After we acknowledge a report, we triage it by severity and target these remedia
**Coordinated disclosure.** We practice coordinated disclosure: we ask that you give us a reasonable
window to ship a fix before any public detail, and we publish details (and credit, if wanted) **once
a fix is available**. We'll keep you updated on progress and agree the disclosure timing with you.
These windows trace to the project's Secure Development Standards (§4.4 RV.2, Appendix A.5).
These windows trace to the project's [Secure Development Standards](../docs/Secure_Development_Standards.md) (§4.4 RV.2, Appendix A.5).

## Dependency (third-party) vulnerabilities

Expand All @@ -61,7 +61,10 @@ is *not* a contradiction of the ≤7-day own-code window above):
- **Exploitation pressure sets priority, not CVSS alone.** We triage **KEV-first** (on CISA's
Known-Exploited-Vulnerabilities list → patch now), then **EPSS** (≥ 0.7 = imminent), with **CVSS only
as a tiebreaker**, and we weigh **reachability** — is the package installed in a shipped profile,
wired into a running graph, and egress-reachable?
wired into a running graph, and egress-reachable? The procedure behind that judgement is
`docs/security/SOUP-DEPENDENCY-HANDLING.md`, a maintainer-internal document;
[`docs/SECURITY-DOCS-POLICY.md`](../docs/SECURITY-DOCS-POLICY.md) explains why it is not published
here and what you can request.

| Class | Trigger | Target (from upstream-fix availability) |
|---|---|---|
Expand All @@ -75,7 +78,8 @@ is *not* a contradiction of the ≤7-day own-code window above):
leave the affected extra uninstalled, or tighten the egress allow-list — and track to the fix. Detection
feeds this lane automatically: blocking `pip-audit`/`npm-audit` against the hash-locked tree, a **daily**
`security.yml` cron (a CVE against an unchanged pin is caught in ~24h), and grouped Dependabot security
PRs.
PRs. The step-by-step response is `docs/security/DEP-CVE-RUNBOOK.md`, also maintainer-internal —
see [`docs/SECURITY-DOCS-POLICY.md`](../docs/SECURITY-DOCS-POLICY.md).

## Scope notes

Expand Down
8 changes: 5 additions & 3 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -150,7 +150,9 @@ scripts/security/scan-tokens.local.txt
# docs/CI-TOPOLOGY.md — STALE: it documents the retired private-repo/public-mirror split and
# scripts/publish/, which the cutover deleted. Publishing it would actively
# mislead. Rewrite for the post-cutover topology or delete it.
# docs/Secure_Development_Standards.md — scans clean, but it was deliberately pulled from the public
# PyPI sdist (#1020); reversing that is an owner call, not a side effect.
# (docs/Secure_Development_Standards.md was here. PUBLISHED 2026-07-29 on the owner's call: the
# document licenses itself as publishable and adopter-facing, messagefoundry.org already served
# it, and 44 citations across the repo — including adr/TEMPLATE.md — pointed at a file that did
# not exist. Published from the newest copy, de-branded back to the generic lineage its own
# revision history names.)
/docs/CI-TOPOLOGY.md
/docs/Secure_Development_Standards.md
12 changes: 7 additions & 5 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,7 @@ without being able to fail.
property that no operator can satisfy on Windows (where the platform read-out is always `null`) — the
same scoping rule as `[security].allowed_client_networks`' companion refusal (ADR 0151). Loopback and
synthetic instances are byte-identical. See
[OFF-LOOPBACK-DEPLOYMENT.md](docs/security/OFF-LOOPBACK-DEPLOYMENT.md) ladder row 12 and
OFF-LOOPBACK-DEPLOYMENT.md ladder row 12 and
[SYSTEM-REQUIREMENTS.md](docs/SYSTEM-REQUIREMENTS.md).
- **Report-only platform memory-encryption read-out on `GET /security/posture`** (ADR 0152 rung 1;
`ENGINE_UI_SEAM` 12 → 13) — `memory_encryption_self_reported_capability` / `..._self_reported_active` /
Expand Down Expand Up @@ -343,7 +343,9 @@ is additive / opt-in.
2026-06-10 full review (v0.2.0 → v0.2.13) surfaced seven verified findings; this release fixes all of
them. No new critical, no SQL injection, no auth bypass, no RCE — the most serious was an
unauthenticated memory-exhaustion DoS in the new default HL7 parser. Each fix ships with a regression
test. See [`docs/reviews/DELTA-REVIEW-2026-07-01.md`](docs/reviews/DELTA-REVIEW-2026-07-01.md).
test. See `docs/reviews/DELTA-REVIEW-2026-07-01.md`, a maintainer-internal document —
[`docs/SECURITY-DOCS-POLICY.md`](docs/SECURITY-DOCS-POLICY.md) states what is withheld and what you
can request.

### Security
- **Bounded the built-in HL7 rich-text repetition escape** (DELTA-01/02;
Expand Down Expand Up @@ -961,12 +963,12 @@ cipher becomes crypto-agile — all additive, with the on-disk `mfenc:v1` format
### Security
- **Dependency fast-response program** — a KEV→EPSS→CVSS triage policy with a **≤72h fast lane** for
actively-exploited dependency CVEs ([`.github/SECURITY.md`](.github/SECURITY.md),
[`docs/security/DEP-CVE-RUNBOOK.md`](docs/security/DEP-CVE-RUNBOOK.md)); a **daily** SCA cron;
`docs/security/DEP-CVE-RUNBOOK.md`); a **daily** SCA cron;
Dependabot moved to the native `uv` ecosystem with **automatic hashed-lock re-export**; **scoped
auto-merge** of safe patches with a **supply-chain cooldown**; weekly **RV.2 metrics**
([`docs/security/DEPENDENCY-METRICS.md`](docs/security/DEPENDENCY-METRICS.md)); and an adopter
(`docs/security/DEPENDENCY-METRICS.md`); and an adopter
remediation SLA + advisory process ([`docs/SUPPORT-POLICY.md`](docs/SUPPORT-POLICY.md),
[`docs/security/ADVISORY-PROCESS.md`](docs/security/ADVISORY-PROCESS.md)).
`docs/security/ADVISORY-PROCESS.md`).
- **Adopter "vulnerable pin" tripwire** — `messagefoundry init`'s scaffolded CI gains an `audit-pin` job
that reds an adopter's build when their pinned engine or its dependencies have a known published
advisory ([`docs/ADOPTER-CI.md`](docs/ADOPTER-CI.md)).
Expand Down
7 changes: 7 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -422,6 +422,13 @@ harness process only.)
- Document each connector/transport and transform with its config schema and an example
message.
- When asked for tabular results, provide the final table directly — not code that generates it.
- **Review security prose by asking what a reader would DO with it, not whether it is accurate.** The
three rules below are instances of it. Reasoning, evidence and dates:
[`docs/Secure_Development_Standards.md`](docs/Secure_Development_Standards.md) §3 *"Reviewing
security prose"* — the source of record.
- **State a load-bearing fact ONCE and link to it; never restate it.**
- **A completeness claim is a liability — prefer "at least" to an enumeration.**
- **A compensating control must not rest on a false premise.**

---

Expand Down
12 changes: 7 additions & 5 deletions docker/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,11 @@
#
# MessageFoundry headless ENGINE image (ADR 0017 "container fast-follow").
#
# The PySide6 console is NEVER in this image — it is a host-side GUI client that reaches the engine
# over the HTTP API (see docs/CONTAINER-EXPOSURE-EVALUATION.md). This image is the headless engine
# only: a pure asyncio/uvicorn/SQLite service with no GUI dependency.
# This image is the headless ENGINE only: a pure asyncio/uvicorn/SQLite service with no GUI dependency
# (see docs/CONTAINER-EXPOSURE-EVALUATION.md). The browser ops console is a separately-versioned optional
# wheel (messagefoundry-webconsole) that is NOT installed here, so a container serve soft-degrades to the
# JSON API only with a warning (ADR 0143). PySide6 is never in the image either — it now backs only the
# standalone test harness, a host-side GUI client that reaches the engine over the HTTP API.
#
# Build the slim default (core + SQLite store):
# docker build -f docker/Dockerfile -t messagefoundry .
Expand All @@ -15,8 +17,8 @@
# docker build -f docker/Dockerfile --target runtime-sqlserver -t messagefoundry:sqlserver .
#
# Both install from per-profile, hash-locked requirements (docker/locks/*.lock) — NOT the all-extras
# requirements.lock, which would drag PySide6/dev tools into a runtime image. The locks are kept in
# sync with uv.lock by the DEP-1 step in .github/workflows/security.yml.
# requirements.lock, which would drag the harness's PySide6 + dev tools into a runtime image. The locks
# are kept in sync with uv.lock by the DEP-1 step in .github/workflows/security.yml.

# Pinned to Debian 12 "bookworm": a mature base whose Microsoft ODBC repo (the -sqlserver variant) is
# well-supported. Debian 13 "trixie" signs its MS repo with a key absent from the legacy microsoft.asc
Expand Down
18 changes: 16 additions & 2 deletions docker/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,22 @@
The **headless engine** runs cleanly as an OCI container (ADR 0017 "container fast-follow"). This
directory holds the image, a Topology-A `compose.yaml`, minimal Kubernetes manifests, and a CI smoke.

The PySide6 **console is never in the image** — it is a host-side GUI client that reaches the engine
over the HTTP API. Only the headless engine (asyncio/uvicorn/SQLite, no GUI dependency) containerizes.
**Only the headless engine containerizes** (asyncio/uvicorn/SQLite, no GUI dependency). The operator
console is the browser web console at `/ui`, a **separately-versioned optional wheel**
(`messagefoundry-webconsole`) that these images do **not** install — a container serve therefore
soft-degrades to the JSON API only, with a warning (ADR 0143); an **explicit**
`[security].serve_web_console=true` against an image without that wheel is refused at startup instead of
degraded, so bake the wheel in before you turn it on. PySide6 is never in the image either: it
now backs only the standalone test harness, a host-side GUI client that reaches the engine over the
HTTP API.

**Concretely: `/ui` is not available on these images.** The wheel build copies only `messagefoundry/`
into the builder ([`Dockerfile`](Dockerfile)), so `messagefoundry_webconsole` is absent from the
runtime venv and the serve path turns the console off in place — **no `/ui` route is ever registered**,
and a browser pointed at it gets nothing. Operate a containerized engine through the **JSON API**, or
add the console wheel in a derived image (the `FROM messagefoundry:<version>` shape under
*Configuration* below) and turn it on there. That is what the images in this directory ship **today**;
it is not a statement about what a future image may carry.

Read [`docs/CONTAINER-EXPOSURE-EVALUATION.md`](../docs/CONTAINER-EXPOSURE-EVALUATION.md) first — it
establishes that the off-loopback security controls are already built; this is packaging + ops wiring.
Expand Down
2 changes: 1 addition & 1 deletion docs/ADOPTER-CI.md
Original file line number Diff line number Diff line change
Expand Up @@ -175,7 +175,7 @@ What CI **does** guarantee on every PR:

## See also

- [SECURING-HANDLER-CONFIG-IN-CI.md](security/SECURING-HANDLER-CONFIG-IN-CI.md) — the handler-security controls (`--strict-handler-security`, the Semgrep taint leg, and Handler-dep `pip-audit`) that harden the `check` + `audit-pin` jobs above (ADR 0144)
- SECURING-HANDLER-CONFIG-IN-CI.md — the handler-security controls (`--strict-handler-security`, the Semgrep taint leg, and Handler-dep `pip-audit`) that harden the `check` + `audit-pin` jobs above (ADR 0144)
- [ADR 0017 — Consumer deployment model](adr/0017-consumer-deployment-model.md)
- [EARLY-ADOPTER-GUIDE.md](EARLY-ADOPTER-GUIDE.md) (§8 validation toolchain, §9 capacity testing)
- [INSTALL-GUIDE.md](INSTALL-GUIDE.md) (manual verify-before-install recipe)
Expand Down
6 changes: 3 additions & 3 deletions docs/AOAG-DEPLOYMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -142,14 +142,14 @@ per-lane msg/s ≈ 1000 / (commit_chain_depth × per-commit round-trip ms)
```

The documented anchor works out to ~7 × 2.84 ms ≈ 20 ms/message ≈ **50 msg/s per ordered
lane** on a LAN-attached SQL Server ([`throughput-roadmap.md`](throughput-roadmap.md)). Three
lane** on a LAN-attached SQL Server ([`throughput-roadmap.md`](archive/throughput/throughput-roadmap.md)). Three
distinct ceilings apply, and you should not conflate them:

- **Per ordered lane:** the commit round-trip chain binds (measured; the box sits ~96% idle at
the single-lane plateau). Every millisecond added to the commit path lands **~7× in every
message**.
- **Per engine box (aggregate, many lanes):** engine CPU binds
([`throughput-build-plan.md`](throughput-build-plan.md)).
([`throughput-build-plan.md`](archive/throughput/throughput-build-plan.md)).
- **Commit-tier IOPS:** headroom on decent hardware (~23,600 commits/s measured on local NVMe —
11–36× above what the engine drove).

Expand Down Expand Up @@ -874,7 +874,7 @@ licensed, which is also why this guide never routes reads at the secondaries. Br
[`DEPLOYMENT.md`](DEPLOYMENT.md) · [`CLOUD-DEPLOYMENT.md`](CLOUD-DEPLOYMENT.md) §2.3 ·
[`CONFIGURATION.md`](CONFIGURATION.md) · [`SERVICE.md`](SERVICE.md) ·
[`SYSTEM-REQUIREMENTS.md`](SYSTEM-REQUIREMENTS.md) · [`THROUGHPUT.md`](THROUGHPUT.md) ·
[`throughput-roadmap.md`](throughput-roadmap.md) ·
[`throughput-roadmap.md`](archive/throughput/throughput-roadmap.md) ·
[`benchmarks/step-b-write-amplification.md`](benchmarks/step-b-write-amplification.md) ·
[`benchmarks/TUNING-BASELINE.md`](benchmarks/TUNING-BASELINE.md) ·
[`EARLY-ADOPTER-GUIDE.md`](EARLY-ADOPTER-GUIDE.md) §14 · ADRs
Expand Down
24 changes: 20 additions & 4 deletions docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,13 +148,28 @@ siblings, and a slow transform never blocks routing.

**Disposition flows with the message, decided by the store finalizer** (count-and-log): `RECEIVED` at
ingress → `ROUTED` / `UNROUTED` after the router → `PROCESSED` (all delivered) / `FILTERED` (every
handler ran, delivered nothing) / `ERROR` (dead-lettered at any stage) once nothing is still in
flight. The finalizer is the **single authority** — it never finalizes while any earlier-stage row is
handler ran, delivered nothing) / `NOT_DEPLOYED` (every destination the handlers addressed is in the
graph but `deployed=false`) / `ERROR` (dead-lettered at any stage) once nothing is still in flight.
The finalizer is the **single authority** — it never finalizes while any earlier-stage row is
pending, so a delivered handler can't mark a message done while a sibling handler's routed row still
awaits transform. The ACK means *receipt-and-persistence*, not a final disposition: a routing/
transform failure is post-ACK, so it is logged + dead-lettered (operators rely on the disposition +
AlertSink), not NAK'd.

`NOT_DEPLOYED` (BACKLOG #233, [ADR 0111](adr/0111-not-deployed-connections.md)) is the one
disposition the finalizer cannot read off the queue rows: it decides `FILTERED` **by absence** (no
rows left, message still `ROUTED`), so a declined destination would otherwise collapse to `FILTERED`
and misreport operator intent. The decline is therefore persisted where the finalizer can see it —
the **transform worker** drops a `Send` whose target outbound is `deployed=false` *before* it becomes
an outbound-stage row, and writes one `not_deployed` `message_events` row per declined destination in
that same handoff transaction (`MessageStore.transform_handoff`). Zero deliveries **with** that event
finalize `NOT_DEPLOYED`; without it, `FILTERED`. A deployed sibling that delivers still finalizes
`PROCESSED`, with the event row carrying the skipped leg. That event is in the store's audit floor,
so the `message_events` verbosity gate can never thin it away — dropping it would take the
disposition with it. Dry-run and the Test Bench never run the finalizer, so a fully-declined message
previews there as `FILTERED`: the declined names stop at `RouteOutcome.declined` and are not carried
into `DryRunResult`.

The router/transform split was taken after Step A's measured write amplification (now ~3 durable
transactions/message for a single-handler message; +1 per extra handler) — recorded in
[docs/benchmarks/step-b-write-amplification.md](benchmarks/step-b-write-amplification.md). An optional
Expand Down Expand Up @@ -186,7 +201,7 @@ flowchart TB
FIN{{"Store finalizer<br/>single disposition authority"}}:::disp
D1["RECEIVED"]:::disp
D2["ROUTED / UNROUTED"]:::disp
D3["PROCESSED / FILTERED / ERROR"]:::disp
D3["PROCESSED / FILTERED<br/>NOT_DEPLOYED / ERROR"]:::disp

SRC --> LISTEN
LISTEN -->|"decode/parse/validate fail"| NAK
Expand Down Expand Up @@ -278,7 +293,8 @@ Messages contain PHI. Access control and the *data* protections are tracked sepa
[SECURITY.md](SECURITY.md) (identity, RBAC, action audit) and [PHI.md](PHI.md) (data-at-rest,
transport, logging, retention, de-identification), which is the authoritative built-vs-planned map.
The per-interface trust boundaries and STRIDE threats are in
[security/THREAT-MODEL.md](security/THREAT-MODEL.md) (PW.1–2 / ASVS V15).
security/THREAT-MODEL.md (PW.1–2 / ASVS V15), a maintainer-internal document —
[SECURITY-DOCS-POLICY.md](SECURITY-DOCS-POLICY.md) explains what is withheld and what you can request.

**Built today:** authentication + RBAC (PHI views gated by `messages:view_raw` /
`messages:view_summary`), a user-attributed append-only **audit log** (hash-chained) of who
Expand Down
Loading
Loading