security: a contract could send a product's credential to any host it named

CHANGELOG.md

@@ -7,6 +7,68 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Security
+
+- **A contract could have a product's database credential delivered to any host it named.** On the
+  `kafka-connect` path, and on the **default** `bring-your-own` / `managed` modes of `debezium` and
+  `airbyte`, forge-cli never connects to the source database: it resolves the contract's
+  `connection.secretRef`, puts the plaintext into the connector config (`database.password`,
+  `connection.password`) and POSTs the whole config to
+  `properties.<engine>.deployment.server_url` — a URL the **contract** chooses. Nothing constrained
+  that host, so the credential could be sent anywhere, needing no allowlist entry and no operator
+  mistake; a loopback `connection.host` even made the build look local. A **public** orchestrator
+  host must now be named in `FLUID_ACQUISITION_DEPLOYMENT_HOST_ALLOWLIST`.
+
+  Loopback and private hosts need no entry: an orchestrator on the cluster network is the ordinary
+  deployment, a credential that stays inside the organisation is not the leak this closes, and
+  refusing it would have broken every working install on upgrade. A Tailscale or other CGNAT
+  address (`100.64.0.0/10`) counts as private, which it did not at first — the extended
+  non-routable-range list the tree already had was consulted by `util/safe_http` but not by the
+  classifier this policy uses, so such an orchestrator was called public and refused with the wrong
+  reason. Both now read one list, in the tier-0 `_net` leaf.
+
+  Link-local and unspecified addresses are refused outright, allowlist or not, because that is where
+  cloud instance metadata lives. A host that resolves to **both** public and non-public addresses is
+  treated as public: one private record beside a public one is how an unnamed host is made to look
+  internal, and the connection is pinned to the public address regardless.
+
+  The gate runs where each runner reads `deployment.server_url`, which is **before** the connector
+  config is built — and building it is what resolves the `secretRef` — so a refused deployment never
+  fetches the password from its secret store. The REST client constructors keep the same check as
+  the chokepoint for any other caller, and it is re-applied per request, so a host that did not
+  resolve at construction is judged for real once it does.
+
+  A URL the **operator** supplied — `fluid import airbyte --server-url`, or
+  `FLUID_IMPORT_AIRBYTE_URL` — is not second-guessed, since the policy exists because a *contract*
+  can choose where a credential goes. The scheme and metadata-address rules still apply there.
+
+  The variable is operator-only (`credentials/protected_env.py`): `fluid apply` hydrates the
+  environment from `.env`, `.env.<env>` and `FLUID_SECRETS_FILE`, which are committed beside the
+  contract, so without that a contract repository could grant itself the allowlist meant to
+  constrain it.
+
+  Deliberately **not** modelled on `FLUID_WEBHOOK_HOST_ALLOWLIST`,
+  `FLUID_FEDERATION_HOST_ALLOWLIST` or `FLUID_COMMAND_CENTER_HOST_ALLOWLIST` despite the matching
+  name. Those are SSRF allowlists: unset, they *permit* any public host and refuse private ones, and
+  the Command Center gate still lets a credential reach any public `https` endpoint unlisted.
+  Mirroring them would have reproduced the hole.
+
+### Upgrade notes
+
+- **A contract whose orchestrator is on a public host now fails until the operator allows it.**
+  Set `FLUID_ACQUISITION_DEPLOYMENT_HOST_ALLOWLIST` in the shell, in CI, or in a
+  `FLUID_SECRETS_FILE` outside the project — **not** in a project `.env`, where it is ignored — to
+  the host or its parent domain (`connect.example.com`, or `example.com` to cover subdomains). A
+  host written with a port never matches, so leave the port off. Deployments on `localhost`,
+  `127.0.0.0/8`, `::1` or a private network need no configuration, which is the common case.
+
+### Known limitations
+
+- Two credential paths this policy does not reach, both pre-existing. An object-store URI can carry
+  its own credentials in the URI itself, which is not a `secretRef` and so is not policed; and
+  `{{ env.NAME }}` interpolation inside a contract bypasses `secretRef` entirely. Neither is changed
+  by this release.
+
 ## [0.24.0] — 2026-10-27
 
 ### Fixed

fluid_build/_net.py

@@ -31,6 +31,125 @@
 import ipaddress
 import socket
 
+# CIDR ranges ``ipaddress.is_private`` does NOT flag but which are nonetheless
+# non-routable / not safe to fetch from. Borrowed from requests-hardened
+# (Saleor, BSD-3). Lives here, at tier-0, because BOTH the broad SSRF gate below
+# and ``util.safe_http.is_public_address`` judge addresses and must agree: they
+# did not before, and the acquisition deployment-host policy -- which allows
+# internal hosts and questions public ones -- classified a Tailscale/CGNAT
+# orchestrator as public and refused it with the wrong reason.
+EXTRA_BLOCKED_NETWORKS = tuple(
+    ipaddress.ip_network(cidr)
+    for cidr in (
+        "100.64.0.0/10",  # RFC 6598 — Carrier-Grade NAT (Tailscale, many ISPs)
+        "192.0.0.0/24",  # RFC 6890 — IETF Protocol Assignments (CPython #113171 gap)
+        "192.0.2.0/24",  # RFC 5737 — TEST-NET-1
+        "192.88.99.0/24",  # RFC 7526 — 6to4 relay anycast (deprecated)
+        "198.18.0.0/15",  # RFC 2544 — benchmarking
+        "198.51.100.0/24",  # RFC 5737 — TEST-NET-2
+        "203.0.113.0/24",  # RFC 5737 — TEST-NET-3
+        "240.0.0.0/4",  # RFC 1112 — class E reserved
+        "2001:20::/28",  # RFC 7343 — ORCHIDv2
+        "2002::/16",  # RFC 3056 — 6to4
+        "5f00::/16",  # RFC 9602 — IPv6 Segment Routing
+        "64:ff9b::/96",  # RFC 6052 — Well-Known NAT64
+        "64:ff9b:1::/48",  # RFC 8215 — Local-Use NAT64
+    )
+)
+
+
+def is_public_address(addr: str) -> bool:
+    """True iff ``addr`` is a routable public IP (v4 or v6).
+
+    Handles three traps ``ipaddress`` alone does not:
+      * IPv4-mapped IPv6 (``::ffff:a.b.c.d``) — on Python <3.12 the v6 form does
+        NOT recurse ``is_private`` / ``is_loopback`` into the embedded v4
+        (gh-87105 / bpo-44904, fixed only in 3.12).
+      * Zone-id-bearing v6 (``fe80::1%eth0``) — strip the scope.
+      * Extended CIDR deny-list (:data:`EXTRA_BLOCKED_NETWORKS`).
+
+    An address that does not parse is not public. ``util.safe_http`` re-exports
+    this; it used to own it, and the two judgements must not drift.
+    """
+    if "%" in addr:
+        addr = addr.split("%", 1)[0]
+    try:
+        ip = ipaddress.ip_address(addr)
+    except ValueError:
+        return False
+    if isinstance(ip, ipaddress.IPv6Address) and ip.ipv4_mapped is not None:
+        ip = ip.ipv4_mapped
+    if (
+        ip.is_private
+        or ip.is_loopback
+        or ip.is_link_local
+        or ip.is_multicast
+        or ip.is_reserved
+        or ip.is_unspecified
+    ):
+        return False
+    for network in EXTRA_BLOCKED_NETWORKS:
+        if ip.version == network.version and ip in network:
+            return False
+    return True
+
+
+#: Every host resolves into exactly one of these.
+#:
+#: ``"public"``     — resolved, and EVERY address is publicly routable.
+#: ``"non_public"`` — resolved, and EVERY address is loopback, private,
+#:                    link-local, unspecified, reserved or on the extended
+#:                    deny-list above.
+#: ``"mixed"``      — resolved to BOTH. Not a curiosity: an attacker who names
+#:                    the host answers with one private record beside their
+#:                    public one, and a gate that stops at the first non-public
+#:                    address calls the host internal while the connection is
+#:                    pinned to the public one.
+#: ``"unresolved"`` — DNS failed, or an address came back unparseable.
+HOST_SCOPES = ("public", "non_public", "mixed", "unresolved")
+
+
+def hostname_scope(hostname: str) -> str:
+    """Which of :data:`HOST_SCOPES` ``hostname`` falls into.
+
+    EVERY address in the answer is examined, and a mixed answer is reported as
+    such. :func:`_hostname_is_private` cannot express that: it answers one
+    question ("may I refuse this?") for which ``mixed`` and ``non_public`` are
+    the same, while a gate with the opposite polarity -- one that ALLOWS
+    internal hosts and asks the operator to name public ones -- must treat a
+    mixed answer as public. Collapsing the two is a complete bypass of such a
+    gate: one decoy AAAA record and the credential goes to the public A record
+    ``safe_http`` then pins. Found by review of the deployment-host policy and
+    reproduced before this existed.
+
+    Kept here, beside :func:`is_public_address` and the range list it consults,
+    rather than re-derived by a caller: this module's whole point is that the
+    blocked set is written once.
+    """
+    try:
+        addresses = socket.getaddrinfo(hostname, None)
+    except socket.gaierror:
+        return "unresolved"
+    saw_public = False
+    saw_non_public = False
+    for entry in addresses:
+        ip_str = entry[4][0]
+        try:
+            ipaddress.ip_address(ip_str.split("%", 1)[0])
+        except ValueError:
+            return "unresolved"
+        if is_public_address(ip_str):
+            saw_public = True
+        else:
+            saw_non_public = True
+    if saw_public and saw_non_public:
+        return "mixed"
+    if saw_non_public:
+        return "non_public"
+    # No addresses at all lands here, as it did before: the old loop fell
+    # through to "not private".
+    return "public"
+
 
 def _hostname_is_private(hostname: str) -> bool:
     """Return True when ``hostname`` resolves to a non-public IP.
@@ -59,26 +178,18 @@ def _hostname_is_private(hostname: str) -> bool:
         depend on it without re-introducing the
         ``observability → build_runners → cli → observability``
         cycle that previously broke ``cli/__init__.py`` import-time.
+
+    Expressed in terms of :func:`hostname_scope` so the blocked range list is
+    written once. ``unresolved``, ``non_public`` and ``mixed`` all answer True,
+    which is what the old "any non-public address" loop did.
+
+    One deliberate change: the extended deny-list is now consulted here too, so
+    a CGNAT / 6to4 / NAT64 / TEST-NET host answers True where it used to answer
+    False. That only ever refuses more, never allows more, and it matches what
+    ``safe_http.resolve_and_validate`` already did for the same addresses --
+    the two disagreed before.
     """
-    try:
-        addresses = socket.getaddrinfo(hostname, None)
-    except socket.gaierror:
-        return True
-    for entry in addresses:
-        ip_str = entry[4][0]
-        try:
-            ip = ipaddress.ip_address(ip_str)
-        except ValueError:
-            return True
-        if (
-            ip.is_loopback
-            or ip.is_private
-            or ip.is_link_local
-            or ip.is_unspecified
-            or ip.is_reserved
-        ):
-            return True
-    return False
+    return hostname_scope(hostname) != "public"
 
 
 def _hostname_is_link_local(hostname: str) -> bool: