feat: define Docker network capacity policy

AGENTS.md

@@ -25,7 +25,7 @@ Agents modifying this repository or adapting a project MUST apply these document
 - Keep original project code compatible with the Unlicense.
 - Do not copy GPL-licensed implementation code into this repository.
 - Preserve required notices for third-party code or substantial examples.
-- Never commit credentials, registration tokens, private keys, real environment files, internal host inventories, private IP addresses, or unredacted infrastructure reports.
+- Never commit credentials, registration tokens, private keys, real environment files, internal host inventories, host addresses, service addresses, or unredacted infrastructure reports. The sole address exception is reviewed Docker `default_address_pools[].base` CIDRs in private Git-authored fleet policy because they are required capacity policy. This exception does not allow credentials, SSH details, secrets, VM, storage, or backup identifiers, unrelated infrastructure addresses, or rendered runtime configuration.
 - Use examples and placeholders for organization-specific configuration.
 - Store long-lived GitHub credentials only in a controller or external secret manager.
 - Do not expose long-lived controller credentials to job runner containers.

README.md

@@ -95,6 +95,12 @@ GitHub runner-group policy decides which repositories may schedule work. A share
 | Project repository | Test Dockerfile, services, fixtures, migrations, test plan, `scripts/ci/run.sh` | Fleet controller credentials or host-specific setup |
 | Private installation configuration | Organization settings, repository authorization, logical controller state, capacity budgets, network policy, and required secret names | Secret values, host addresses, project runtime dependencies, or test logic |
 
+One narrow exception lets a private installation configuration commit exact,
+reviewed Docker `default_address_pools[].base` CIDRs. These ranges are allocation
+capacity policy, not host addresses, credentials, host identity, or routable
+service endpoints. VM, storage, backup, SSH, rendered runtime, and unrelated
+infrastructure details remain outside Git.
+
 A public application can use the same fleet indirectly. Its public repository keeps pull-request validation unprivileged, while a separate private delivery repository checks out an approved immutable commit and performs protected CI, release, or deployment work. The public repository itself never receives privileged runner-group access or fleet credentials. See [Public projects, private delivery, and private configuration](docs/PUBLIC-PRIVATE-CONFIGURATION.md).
 
 ## What makes it different?

docs/DESIRED-STATE.md

@@ -13,6 +13,13 @@ Schema v3 makes a reviewed private configuration repository the authority for co
 
 Host addresses, VM IDs, storage names, backup identifiers, SSH details, tokens, private keys, and rendered `.env` files are rejected from the Git-authored configuration.
 
+Private policy has one explicit address exception. Each reviewed
+`docker_network_policy.default_address_pools[].base` CIDR is committed because
+the controller must render and inspect that exact allocation pool. It is capacity
+policy, not a host address, host identity, credential, or routable service
+endpoint. The exception does not admit any other infrastructure address or
+runtime detail.
+
 ## Schema v3
 
 Each runner pool declares:
@@ -32,10 +39,44 @@ Each controller has a unique object key and declares:
 - `experimental`, `stable`, or `retiring` lifecycle;
 - a full pinned ci-fleet engine commit;
 - a zero managed minimum and reviewed maximum runner capacity;
-- CPU cores and memory per ephemeral runner.
+- CPU cores and memory per ephemeral runner;
+- reviewed Docker default-address pools, a positive per-runner network bound,
+  and a reserved subnet count.
 
 Active and drained controllers reserve their configured maximum against the pool budget. A drained controller has zero effective runtime capacity but keeps its reservation, so an undrain cannot silently overcommit the pool. Disabled controllers reserve no capacity.
 
+The Docker network policy uses IPv4 CIDR `base` values and a Docker subnet
+prefix `size` no longer than `/29`, which leaves enough addresses for an
+ordinary Compose network. Validation rejects malformed or overlapping pools,
+allocation prefixes broader than their base, and active or drained policies with fewer subnets than
+`max_runners * networks_per_runner + reserve_subnets + 1`. The final subnet is reserved for the
+persistent controller Compose network. Disabled controllers do not reserve
+runner subnet capacity, but their retained policy must still cover the reserve
+and controller network. Real pool values belong only in the private desired-state
+repository under the narrow address-pool exception above. Public examples use
+RFC 5737 documentation ranges, which strict validation rejects until the operator
+supplies a reviewed operational Docker pool CIDR.
+
+`docker_network_policy` is optional only to preserve a staged upgrade path from
+older schema-v3 engines whose exact-key validator does not recognize it. Upgrade
+an existing controller in three reviewed desired-state commits. First change
+only `engine_ref`. After routine reconciliation shows that exact engine is active,
+record `docker_network_policy_config: true` for that controller and ref in
+`engine-rollout-evidence.json`. Only then add the reviewed network policy without
+changing the engine or evidence. Transition validation reads the evidence from
+the previous integrated state, so a commit that adds evidence and policy together
+cannot satisfy the gate. Do not add the field while the old engine still performs
+reconciliation. Once present, the policy requires current evidence naming the
+selected engine and declaring `docker_network_policy_config: true`; remove the
+policy before selecting an engine without that evidence.
+
+This phase renders the policy solely for read-only health inspection. It does
+not write `daemon.json`, restart Docker, create or remove networks, prune
+resources, drain runners, or alter controller scale. Circuit breaking,
+frequent orphan reconciliation, daemon configuration rollout, transactional
+exhausted-pool recovery, and MTFM/TF2 consumer-label changes remain later
+issue #81 slices.
+
 Managed prewarmed runners are not currently supported: `min_runners` is fixed at zero in schema, semantic validation, rendering, and preflight. This keeps idle privileged workers absent and prevents reviewed configuration from passing validation only to fail host adoption.
 
 The authoritative fictional contract is in [`templates/config-repository`](../templates/config-repository/README.md).

docs/HEALTH-MONITORING.md

@@ -17,6 +17,7 @@ The check covers:
 - root and Docker filesystem space and inodes;
 - available memory, swap use, per-CPU load, and OOM evidence from the last 24 hours;
 - Docker availability, controller state/restarts, and configured versus effective capacity;
+- configured/used/free Docker subnet headroom and legacy networks, without exposing addresses;
 - inactive, unhealthy, restarting, and stale fleet-labelled resources, including week-old build cache;
 - cleanup, drift, health, and update services/timers;
 - failed package state, pending reboot, and clock synchronization;
@@ -25,6 +26,20 @@ The check covers:
 
 It reports but never prunes, restarts, or repairs resources. Project source, logs, environment values, tokens, and private keys are never included.
 
+Docker network inspection is read-only. Healthy headroom is reported when free
+subnets remain above the reviewed reserve, low water and legacy/nonconforming
+networks are warnings, and exhaustion is critical. A malformed policy or failed
+Docker network listing/inspection is critical rather than falsely healthy.
+When policy parsing succeeds during an inspection outage, the local snapshot
+retains the configured subnet count and reserve without inventing usage counts.
+Status reports include aggregate configured, used, free, and legacy counts only
+after a successful measurement; otherwise they omit the optional network field.
+
+This detection-only phase does not mutate the Docker daemon or networks.
+Controller circuit breaking, frequent orphan reconciliation, daemon policy
+application, transactional recovery, cleanup, and consumer-label migrations
+remain later issue #81 work.
+
 ## Threshold overrides and hooks
 
 Defaults are intentionally conservative: disk and inode warning/critical at 80/90%, available memory warning/critical at 15/8%, sustained swap use under five-minute memory pressure warning/critical at 25/50%, per-CPU fifteen-minute load warning/critical at 1.0/1.5, and controller restart warning at 3.

docs/STATUS-REPORTING.md

@@ -24,12 +24,17 @@ The machine-readable contract is `schemas/status-report-v1.json`. It reports:
 - reconciliation, drift, health, and cleanup timer states;
 - current, busy, and configured-maximum runner counts;
 - CPU use, logical CPU count, memory, swap, root/Docker disk and inode use, and 1/5/15-minute load;
-- Docker availability and OOM evidence;
+- Docker availability, OOM evidence, and optional measured configured/used/free/legacy subnet counts;
 - one controlled error code/message, report generation time, and schema version.
 
 All times are Unix seconds. Commit values are empty when unavailable. Receiver validation rejects unknown fields and unsupported schema versions rather than guessing at compatibility.
 
 `error.message` is derived only from a controlled error code (`_` becomes a space). Raw exception text is never transmitted.
+Docker pool prefixes and network addresses are intentionally absent from the
+status contract; they remain in private desired state and host-local inspection.
+The reporter omits the optional `docker.network` aggregate when Docker network
+inspection is unavailable. It does not publish zero usage or capacity as a
+substitute for missing measurements.
 
 ## Authentication
 

engine-capabilities.json

@@ -1,6 +1,7 @@
 {
   "schema_version": 1,
   "capabilities": {
+    "docker_network_policy_config": true,
     "status_reporting_config": true,
     "required_status_reporting": true
   }

schemas/status-report-v1.json

@@ -102,7 +102,19 @@
     },
     "docker": {
       "type": "object", "additionalProperties": false, "required": ["healthy", "oom"],
-      "properties": {"healthy": {"type": "boolean"}, "oom": {"type": "boolean"}}
+      "properties": {
+        "healthy": {"type": "boolean"},
+        "oom": {"type": "boolean"},
+        "network": {
+          "type": "object", "additionalProperties": false, "required": ["configured", "used", "free", "legacy"],
+          "properties": {
+            "configured": {"type": "integer", "minimum": 0},
+            "used": {"type": "integer", "minimum": 0},
+            "free": {"type": "integer", "minimum": 0},
+            "legacy": {"type": "integer", "minimum": 0}
+          }
+        }
+      }
     },
     "error": {
       "oneOf": [

scripts/desired_state.py

@@ -5,6 +5,7 @@
 
 import argparse
 import importlib.util
+import ipaddress
 import json
 import os
 import re
@@ -30,6 +31,7 @@
 HOST_OPTIONAL = {"CI_FLEET_RUNNER_TTL"}
 REQUIRED_STATUS_CAPABILITY = "required_status_reporting"
 STATUS_REPORTING_CONFIG_CAPABILITY = "status_reporting_config"
+DOCKER_NETWORK_POLICY_CONFIG_CAPABILITY = "docker_network_policy_config"
 
 
 class DesiredStateError(ValueError):
@@ -137,6 +139,63 @@ def validate_host_values(values: dict[str, str]) -> dict[str, str]:
     }
 
 
+def validate_docker_network_policy(policy: dict[str, Any], *, path: str, max_runners: int) -> tuple[int, int, int, list[dict[str, Any]]]:
+    if not isinstance(policy, dict):
+        raise DesiredStateError(f"{path}: must be an object")
+    required = {"default_address_pools", "networks_per_runner", "reserve_subnets"}
+    if set(policy) != required:
+        unknown = sorted(set(policy) - required)
+        missing = sorted(required - set(policy))
+        messages: list[str] = []
+        if missing:
+            messages.append(f"missing keys: {', '.join(missing)}")
+        if unknown:
+            messages.append(f"unknown keys: {', '.join(unknown)}")
+        raise DesiredStateError(f"{path}: " + "; ".join(messages))
+    reserve = policy.get("reserve_subnets")
+    if type(reserve) is not int or reserve < 1:
+        raise DesiredStateError(f"{path}.reserve_subnets: must be a positive integer")
+    networks_per_runner = policy.get("networks_per_runner")
+    if type(networks_per_runner) is not int or networks_per_runner < 1:
+        raise DesiredStateError(f"{path}.networks_per_runner: must be a positive integer")
+    pools = policy.get("default_address_pools")
+    if type(pools) is not list or not pools:
+        raise DesiredStateError(f"{path}.default_address_pools: must be a non-empty list")
+    parsed: list[dict[str, Any]] = []
+    for index, pool in enumerate(pools):
+        pool_path = f"{path}.default_address_pools[{index}]"
+        if not isinstance(pool, dict) or set(pool) != {"base", "size"}:
+            raise DesiredStateError(f"{pool_path}: must contain only base and size")
+        base = pool.get("base")
+        size = pool.get("size")
+        if not isinstance(base, str):
+            raise DesiredStateError(f"{pool_path}.base: must be a CIDR prefix")
+        if type(size) is not int or size < 0 or size > 29:
+            raise DesiredStateError(f"{pool_path}.size: must be an IPv4 prefix length between 0 and 29")
+        try:
+            network = ipaddress.ip_network(base, strict=True)
+        except ValueError as exc:
+            raise DesiredStateError(f"{pool_path}.base: malformed IPv4 prefix") from exc
+        if network.version != 4:
+            raise DesiredStateError(f"{pool_path}.base: malformed IPv4 prefix")
+        if size < network.prefixlen:
+            raise DesiredStateError(f"{pool_path}.size: impossible subnet count for {base}")
+        parsed.append({"base": base, "network": network, "size": size})
+    for left, item in enumerate(parsed):
+        for right in range(left + 1, len(parsed)):
+            other = parsed[right]
+            if item["network"].overlaps(other["network"]):
+                raise DesiredStateError(
+                    f"{path}.default_address_pools[{left}].base: overlaps configured pool {right}"
+                )
+    configured = sum(1 << (item["size"] - item["network"].prefixlen) for item in parsed)
+    if configured < max_runners * networks_per_runner + reserve + 1:
+        raise DesiredStateError(
+            f"{path}: network capacity cannot satisfy max_runners * networks_per_runner + reserve_subnets + one controller Compose network"
+        )
+    return configured, reserve, networks_per_runner, parsed
+
+
 def select_controller(config: dict[str, Any], controller_id: str) -> tuple[dict[str, Any], dict[str, Any]]:
     controllers = config["controllers"]
     if controller_id not in controllers:
@@ -169,6 +228,17 @@ def build_rendered_env(
     state = controller["state"]
     configured_max = controller["max_runners"]
     effective_max = configured_max if state == "active" else 0
+    network_policy_configured = "docker_network_policy" in controller
+    network_policy = controller.get("docker_network_policy")
+    configured_subnets, reserve_subnets, networks_per_runner, parsed_pools = (0, 0, 0, [])
+    if network_policy_configured:
+        configured_subnets, reserve_subnets, networks_per_runner, parsed_pools = validate_docker_network_policy(
+            network_policy,
+            path=f"$.controllers.{controller_id}.docker_network_policy",
+            max_runners=configured_max if state != "disabled" else 0,
+        )
+        if DOCKER_NETWORK_POLICY_CONFIG_CAPABILITY not in (engine_capabilities or set()):
+            raise DesiredStateError("selected engine does not support Docker network policy configuration")
     short_commit = engine_commit[:12]
     rendered = {
         "CI_FLEET_CAPACITY_BUDGET": str(pool["capacity_budget"]),
@@ -194,6 +264,13 @@ def build_rendered_env(
         "CI_FLEET_VERSION": short_commit,
         **validate_host_values(host_values),
     }
+    if network_policy_configured:
+        rendered["CI_FLEET_DOCKER_DEFAULT_ADDRESS_POOL_COUNT"] = str(len(parsed_pools))
+        rendered["CI_FLEET_DOCKER_NETWORKS_PER_RUNNER"] = str(networks_per_runner)
+        rendered["CI_FLEET_DOCKER_NETWORK_RESERVE_SUBNETS"] = str(reserve_subnets)
+        for index, pool_config in enumerate(parsed_pools):
+            rendered[f"CI_FLEET_DOCKER_DEFAULT_ADDRESS_POOL_{index}_BASE"] = pool_config["base"]
+            rendered[f"CI_FLEET_DOCKER_DEFAULT_ADDRESS_POOL_{index}_SIZE"] = str(pool_config["size"])
     reporting_configured = "status_reporting" in controller
     reporting_required = (controller.get("status_reporting") or {}).get("enabled") is True
     if reporting_required and REQUIRED_STATUS_CAPABILITY not in (engine_capabilities or set()):
@@ -222,6 +299,11 @@ def build_rendered_env(
         "engine_repository": config["organization"]["delivery_engine"],
         "status_reporting_configured": reporting_configured,
         "status_reporting_required": reporting_required,
+        "docker_network_policy_configured": network_policy_configured,
+        "docker_network_default_address_pools": len(parsed_pools),
+        "docker_networks_per_runner": networks_per_runner,
+        "docker_network_reserve_subnets": reserve_subnets,
+        "docker_network_configured_subnets": configured_subnets,
     }
     return rendered, metadata
 
@@ -288,6 +370,8 @@ def command_engine(args: argparse.Namespace) -> None:
 
 def command_validate_engine_capabilities(args: argparse.Namespace) -> None:
     capabilities = load_engine_capabilities(args.manifest)
+    if args.require_docker_network_policy_config and DOCKER_NETWORK_POLICY_CONFIG_CAPABILITY not in capabilities:
+        raise DesiredStateError("selected engine does not support Docker network policy configuration")
     if args.require_status_reporting_config and STATUS_REPORTING_CONFIG_CAPABILITY not in capabilities:
         raise DesiredStateError("selected engine does not support status reporting configuration")
     if args.require_status_reporting and REQUIRED_STATUS_CAPABILITY not in capabilities:
@@ -327,6 +411,7 @@ def parse_args() -> argparse.Namespace:
 
     capabilities = subparsers.add_parser("validate-engine-capabilities", help="validate an engine capability declaration")
     capabilities.add_argument("--manifest", type=Path, required=True)
+    capabilities.add_argument("--require-docker-network-policy-config", action="store_true")
     capabilities.add_argument("--require-status-reporting-config", action="store_true")
     capabilities.add_argument("--require-status-reporting", action="store_true")
     capabilities.set_defaults(function=command_validate_engine_capabilities)