Network/env follow-ups (#177) and runner-owned adapter deadline (#175)

change_log.md

@@ -3,6 +3,7 @@
 ## Unreleased
 
 ### Breaking
+* Network and env hardening follow-ups (#177) and runner-owned adapter deadline (#175). Breaking: (1) `process:env=` now refuses one single-sourced table (`capability.CodeLoadingEnvNames` + `CodeLoadingEnvPrefixes`, rendered into generated Go and JavaScript) that adds `PATH`, `HOME`, `XDG_CONFIG_HOME`/`XDG_CONFIG_DIRS`/`XDG_DATA_HOME`/`XDG_DATA_DIRS`, every `GIT_*`, `BASH_FUNC_*` and `LUA_*` name, `SHELLOPTS`, `BASHOPTS`, `PS4`, `PROMPT_COMMAND`, `IFS`, `CDPATH`, `GLOBIGNORE`, `ZDOTDIR`, `PYTHONHOME`, `PYTHONUSERBASE`, `PYTHONINSPECT`, `PYTHONWARNINGS`, `PYTHONBREAKPOINT`, `PYTHONEXECUTABLE`, `NODE_EXTRA_CA_CERTS`, `NODE_TLS_REJECT_UNAUTHORIZED`, `NODE_REPL_EXTERNAL_MODULE`, `PERLLIB`, `PERL5DB`, `GOFLAGS`, `GODEBUG`, `GOROOT`, `GOPATH`, `GOTOOLCHAIN`, `GOENV`, `JAVA_TOOL_OPTIONS`, `_JAVA_OPTIONS`, `JDK_JAVA_OPTIONS`, `CLASSPATH`, `GCONV_PATH`, `LOCPATH`, `NLSPATH`, `HOSTALIASES`, `RESOLV_HOST_CONF`, `SSL_CERT_FILE`, `SSL_CERT_DIR`, `CURL_CA_BUNDLE` and `REQUESTS_CA_BUNDLE`; grants such as `process:env=HOME` or `process:env=PATH` that used to parse now fail at parse time (interpreter, `-run-bc`) and are skipped by generated Go/JS. (2) On a redirect that changes scheme, host or effective port, every request header except `capability.RedirectSafeHeaders` (`Accept`, `Accept-Encoding`, `Accept-Language`, `Content-Type`, `Content-Length`, `Content-Encoding`, `User-Agent`) is dropped, including `Proxy-Authorization`, `Referer` and custom auth headers; previously only `Authorization` and `Cookie` were. (3) Redirect targets carrying userinfo are refused on all three targets. (4) The `localhost` opt-out now requires every resolved address to be loopback (127/8, `::1`). (5) `3fff::/20` and `5f00::/16` join `capability.SpecialPrefixes`, and IPv4-translated `::ffff:0:a.b.c.d` is decoded and checked as IPv4. (6) Node vets and pins the whole resolved address set rather than the first entry, and treats `[::ffff:a.b.c.d]` as `a.b.c.d` like core and generated Go (a grant for one now matches the other; same-literal redirects keep headers). (7) #175: `vm.ExecutionPolicy.AdapterDeadline` lets the runner raise the model-adapter wall-clock budget from the 1s default up to `vm.MaxAdapterDeadline` (30s); larger values are clamped, a shorter `Deadline` still wins, and `vm.AdapterDeadlineFlag` / `vm.ParseAdapterDeadline` give runners a `-adapter-deadline` flag that rejects zero, negative and over-cap values. Candidates cannot set it, and artifact runs are unaffected. New parity tests check that the generated Go and JS tables equal the capability tables, and the blocked/public address lists now also run against generated Go. Pins, freeze, tip-lock, arity and target parity are unchanged. Journal: [Network/env follow-ups and adapter deadline](docs/journals/2026-10-13_network_env_followups_adapter_deadline.md).
 * Network destination hardening (#171, #172). On the capability client, generated Go and Node JavaScript, hostname grants now refuse DNS results in loopback (127/8, ::1), RFC1918 (10/8, 172.16/12, 192.168/16), ULA (fc00::/7), NAT64 (64:ff9b::/96), link-local and unspecified ranges at connection time. Explicit IP-literal grants and the exact name `localhost` opt out; allowed bytecode receipt effects record `private_destination_opt_out: "explicit IP-literal or localhost grant"`. Multicast (224/4, ff00::/8) is always refused, including explicit literals. Review follow-up (#176, part of #177): the blocked set is one shared table (`capability.SpecialPrefixes`) rendered into generated Go and JavaScript. It adds CGNAT 100.64/10 (including 100.100.100.200 metadata), 0/8, 192.0.0/24, 198.18/15, 240/4 (including broadcast), the documentation ranges, NAT64 64:ff9b:1::/48, Teredo 2001::/32, 100::/64, fec0::/10 and any non-global-unicast address. IPv4 inside IPv4-compatible (::a.b.c.d) and 6to4 (2002::/16) addresses is decoded and checked. A redirect may land on an IP-literal or `localhost` destination only when the request started on that same host: a hostname grant can no longer bounce into a granted private IP or localhost, and localhost cannot bounce into 127.0.0.1. HTTPS-to-HTTP redirects are refused even when both destinations are granted; Authorization and Cookie are stripped after any host, port or scheme change for the rest of the redirect chain. Ten-hop redirect caps and ignored HTTP_PROXY/HTTPS_PROXY are regression-tested on core, generated Go and JS; Node uses native HTTP requests with the checked DNS address pinned and no proxy agent, including under Node 24's NODE_USE_ENV_PROXY. JavaScript previously accepted short IPv4 grants such as `network:fetch=127.1` through WHATWG canonicalization; it now refuses ambiguous numeric grants and URLs like Go. Trailing-dot names report an invalid hostname rather than an ambiguous numeric host. Browser JavaScript retains host/redirect checks but cannot inspect DNS results. Migration: use an explicit IP-literal or `localhost:<port>` grant only when private access is intended; granting a route its own localhost port also grants access to its `/admin`. `process:env=` now also refuses PYTHONPATH, PYTHONSTARTUP, NODE_OPTIONS, NODE_PATH, BASH_ENV, ENV, PERL5LIB, PERL5OPT, RUBYOPT and RUBYLIB, case-insensitively, alongside LD_*/DYLD_*; generated children omit them even if a raw grant bypasses parsing. `apps/webhook_relay` still reads UPSTREAM_URL at runtime but its route remains pinned to 127.0.0.1:8099; changing the environment does not expand its authority. No production HFIR flip; #90 stays Partial; pins, freeze tests and target exclusions unchanged. Reference: `docs/reference/network_authority.md`; evidence: `docs/journals/2026-10-13_network_destination_hardening.md`.
 * HFIR model adapter resource bounds (#169, #170): shared closed scalar `const`/`binary` subexpressions now lower once into private functions instead of exponentially expanded fragments; effects, variable reads and allocating expressions retain their evaluation sites and order. Candidates exceeding 2048 dependency levels, 65536 expanded nodes or 4194304 aggregate cached-fragment nodes fail closed with `HFIR_TRANSPORT_TOO_DEEP` / `HFIR_TRANSPORT_TOO_LARGE`, including callers that construct `Candidate` directly. Breaking for adapter consumers: in-memory programs returned by `CompileCandidate` now have a maximum one-second wall-clock budget (a shorter runner deadline still applies); a sleep beyond the remaining budget is refused immediately with `HFIR_DEADLINE_EXCEEDED`, which `try_let` cannot catch. Receipt paths report both the effective budget and typed failure. The adapter constraint is runner-owned and ephemeral; HFBC serialization remains unchanged, and artifact runners still select their execution policy. Explicit-deadline standalone VM sleeps also refuse durations beyond their remaining budget with the existing `LIMIT_EXCEEDED` code. Production HFIR compilation, #90 (Partial), target fences, SHA-256 pins, freeze tests and the Assurance tip-lock remain unchanged. Journal: [HFIR adapter resource blockers](docs/journals/2026-10-13_hfir_adapter_resource_blockers.md).
 * HFIR model adapter (#88) now covers the full `-compile-hfir-bc` language. `hfir-model-adapter/v1` accepts every node kind `LowerToBytecode` lowers from source (`hfir.AdapterNodeKinds`): `defun`/`param`/`call`/`return`, `while`, `for`, `try`/`catch`, `read_file`, `write_file`, `mkdir`, `exec`, `fetch`, `parse_json`, `is_nil`, `map_keys`, `html_escape`, `attr_escape`, `regex_match`, `time_now`, `sleep` (transport role `duration`), `cli_args`, `req_query`/`req_header`/`req_path`, and `(map_get <expr> key)` via a `target` edge, on top of the Phase-1 kinds. `for`/`while` control edges are derived from validated roles like `if`; the schema still has no control-edge field. Kinds the source lowerer does not lower (`spawn`, `spawn_agent`, `grants`, `llm_generate`, `http_server`/`res*`, store/db forms, `read_line`, `http_request`, `match`, ...) still fail with `HFIR_TRANSPORT_KIND`. No new opcode, capability, grant, lowering or language form; effects reached through the adapter are checked by the same run-time grants and fail closed without them. Breaking for adapter clients: candidate bounds rise to 4096 nodes, 64 inputs per node and 1 MiB per transport (from 128 / 32 / 64 KiB); the transport type check now types BOOL consts as bool, `to_int`/`to_float` as numbers and `str_split` as a list, and accepts string `+` string, so a candidate mixing a bool into arithmetic, or a `sleep` whose duration is a known non-number, is now rejected with `HFIR_TRANSPORT_TYPE`; a `return` outside a `defun` body is rejected with `HFIR_TRANSPORT_RETURN` unless it is a straight-line main-level statement (program/sequence/let body or if branch), which source already allows; cycles are rejected (`HFIR_TRANSPORT_CYCLE`) before type inference; non-const nodes never carry `literal_kind`. New experimental flag `-hfir-adapter-roundtrip` (with `-compile-hfir-bc`) sends the source graph through the adapter (encode, strict decode, `CompileCandidate`). `TestModelAdapterCoversArityTable` requires every one of the 386 corpus programs `-compile-hfir-bc` lowers to round-trip byte for byte, and every arity-table form to be covered (126 of 159) or carry an exemption with a rejected sample (33) that the test checks stays outside `-compile-hfir-bc`. The prod-flip criteria's model-adapter row is updated; the promote-blocker fence, production `-compile-bc`, #90 (Partial) and the tip-lock are unchanged. Pinned token benchmarks unchanged.

docs/journals/2026-10-13_network_env_followups_adapter_deadline.md

@@ -0,0 +1,22 @@
+# Network/env follow-ups and runner-owned adapter deadline (#177, #175)
+
+Consumer-driven findings from the #176 reviews (Lain, Motoko) and the #173
+review (Motoko). Each item has a negative test that was mutation-checked:
+the fix was reverted or weakened locally and the named test failed.
+
+| Finding | Fix | Negative test |
+| --- | --- | --- |
+| env refusal list missed `GIT_*`, `GCONV_PATH`, `PYTHONHOME`, `PERL5DB`, `JAVA_TOOL_OPTIONS`, `SHELLOPTS`/`BASHOPTS`/`PS4`, `HOME`/`XDG_CONFIG_HOME`, `PATH`, and was duplicated in three files | One table, `capability.CodeLoadingEnvNames` + `CodeLoadingEnvPrefixes`, rendered into generated Go and JS | `nettest.EnvRefused`/`EnvAllowed` run on core (`TestEnvGrantRefusesCodeLoading`), generated Go (`TestCodeLoadingEnv`) and Node (`TestJSCodeLoadingEnv`); `TestSharedTables` / `TestJSSharedTables` compare the rendered tables |
+| `Proxy-Authorization` and custom auth headers survived an origin change | Allow-list `capability.RedirectSafeHeaders`; every other header is dropped | `CrossOriginHeaders` (core, generated Go), `TestJSCrossOriginHeadersAndUserinfo` |
+| userinfo in redirect targets was followed | Refused in `CheckRedirect` and the Node hop loop | `RedirectUserinfo`, `TestJSCrossOriginHeadersAndUserinfo` |
+| Node pinned only the first resolved address | The whole vetted set is pinned; `lookup` honors `all` and `family` | `TestJSLocalhostLoopbackAndPinAll` |
+| core and Node disagreed on `[::ffff:a.b.c.d]` | Node canonicalizes mapped literals like `canonicalIP` | `nettest.MappedGrantCases` on all three targets |
+| `::ffff:0:0/96` not decoded; `3fff::/20`, `5f00::/16` missing | Decode and add rows | new `BlockedAddresses` / `PublicAddresses` rows; `TestAddressLists` now runs them on generated Go |
+| `localhost` opt-out could reach any address | Every resolved address must be loopback | `LocalhostLoopbackOnly`, `TestJSLocalhostLoopbackAndPinAll` |
+| #175: adapter budget fixed at 1s | `ExecutionPolicy.AdapterDeadline`, clamped at 30s; `AdapterDeadlineFlag` rejects over-cap values | `TestAdapterDeadlineRunnerOwned`, `TestAdapterDeadlineFlag`, `TestAdapterDeadlineIgnoredForArtifacts` |
+
+Choice: the env list stays a deny-list (single-sourced) rather than an
+allow-list, because existing grants for ordinary variables (`LANG`, `TZ`,
+app tokens) must keep working; `HOME` and `PATH` are now refused, which is
+the one consumer-visible break. Exec children are already granted by
+absolute path and start with an empty environment.

docs/reference/network_authority.md

@@ -47,24 +47,34 @@ capability at all: they only touch the request the server already accepted.
   range in the shared table `capability.SpecialPrefixes`: CGNAT
   (`100.64/10`, which includes the `100.100.100.200` metadata address),
   `0/8`, `192.0.0/24`, `198.18/15`, `240/4` (including broadcast), the
-  documentation ranges, `64:ff9b:1::/48`, Teredo (`2001::/32`), `100::/64`
-  and `fec0::/10`. Generated Go and JavaScript render the same table.
+  documentation ranges, `64:ff9b:1::/48`, Teredo (`2001::/32`), `100::/64`,
+  `fec0::/10`, `3fff::/20` and `5f00::/16`. Generated Go and JavaScript render the same table.
   IPv4-mapped IPv6 is checked as IPv4, and IPv4 embedded in IPv4-compatible
-  (`::a.b.c.d`) and 6to4 (`2002::/16`) addresses is decoded and checked. Go checks the
-  resolved address before dialing and again at socket creation. Node pins
-  its connection to the address checked by its lookup.
+  (`::a.b.c.d`), IPv4-translated (`::ffff:0:a.b.c.d`) and 6to4 (`2002::/16`)
+  addresses is decoded and checked. Go checks every resolved address before
+  dialing and again at socket creation. Node vets every address its lookup
+  returns and pins the connection to that whole vetted set (#177).
 * Explicit IP-literal grants and the exact name `localhost` are deliberate
-  opt-outs for those ranges. They still require the exact host and port;
+  opt-outs for those ranges. `localhost` reaches loopback only: if it
+  resolves to any non-loopback address the fetch is refused (#177). They
+  still require the exact host and port;
   `localhost` does not authorize `127.0.0.1`, or vice versa. Multicast
   (`224.0.0.0/4`, `ff00::/8`) is always refused, even with a literal grant.
 * A redirect may land on an IP-literal or `localhost` destination only if
   the request started on that same host. A hostname grant can't be
   redirected into a granted private IP or `localhost`, and `localhost`
   can't be redirected to `127.0.0.1` (or the reverse).
+* An IPv4-mapped literal `[::ffff:a.b.c.d]` is the same host as `a.b.c.d`
+  on every target, for grant matching, opt-out redirects and origin
+  comparison (#177).
+* A redirect target carrying userinfo (`user:pw@` or `user@`) is refused.
 * HTTPS-to-HTTP redirects are refused even when both hosts are granted.
-  On any host, effective port or scheme change, Authorization and Cookie
-  headers are removed for the rest of the chain, including subsequent hops
-  on the new origin.
+  On any host, effective port or scheme change, every request header except
+  the shared `capability.RedirectSafeHeaders` (`Accept`, `Accept-Encoding`,
+  `Accept-Language`, `Content-Type`, `Content-Length`, `Content-Encoding`,
+  `User-Agent`) is removed for the rest of the chain, including subsequent
+  hops on the new origin. This drops `Authorization`, `Proxy-Authorization`,
+  `Cookie` and custom auth headers such as `X-Api-Key` (#177).
 * `HTTP_PROXY` / `HTTPS_PROXY` (and lowercase variants) are ignored by
   governed fetches. Node 24 with `NODE_USE_ENV_PROXY=1` honors proxy env in
   its native fetch; the governed Node path uses native HTTP requests with

docs/reference/process_authority.md

@@ -118,14 +118,24 @@ Rules, enforced identically on the interpreter (`-run`), `-run-bc` (both
 
 ## Environment pass-through and receipts (#166)
 
-* `process:env=NAME` refuses dynamic-loader variables: any name starting with
-  `LD_` (including `LD_PRELOAD`, `LD_LIBRARY_PATH`, `LD_AUDIT`) or `DYLD_`,
-  plus interpreter code-loading variables `PYTHONPATH`, `PYTHONSTARTUP`,
-  `NODE_OPTIONS`, `NODE_PATH`, `BASH_ENV`, `ENV`, `PERL5LIB`, `PERL5OPT`,
-  `RUBYOPT` and `RUBYLIB`, compared case-insensitively. Passing one to an allow-listed program would
-  let it load arbitrary code. The grant is rejected at parse time on the
-  interpreter and `-run-bc`; generated Go and JavaScript skip such names
-  when building the child environment.
+* `process:env=NAME` refuses code-loading, startup, config-lookup, trust-store
+  and program-search variables, compared case-insensitively. The one table is
+  `capability.CodeLoadingEnvNames` plus the prefixes in
+  `capability.CodeLoadingEnvPrefixes` (`LD_*`, `DYLD_*`, `GIT_*`,
+  `BASH_FUNC_*`, `LUA_*`); generated Go and JavaScript render the same tables
+  and a parity test runs one independent list against all three (#177). It
+  includes `PATH`, `HOME`, `XDG_CONFIG_HOME`/`XDG_CONFIG_DIRS`, shell options
+  (`SHELLOPTS`, `BASHOPTS`, `PS4`, `PROMPT_COMMAND`, `IFS`), Python
+  (`PYTHONPATH`, `PYTHONHOME`, `PYTHONSTARTUP`, ...), Node (`NODE_OPTIONS`,
+  `NODE_PATH`, `NODE_EXTRA_CA_CERTS`), Perl (`PERL5LIB`, `PERLLIB`,
+  `PERL5OPT`, `PERL5DB`), Ruby, Go (`GOFLAGS`, `GODEBUG`, ...), Java
+  (`JAVA_TOOL_OPTIONS`, `_JAVA_OPTIONS`, `JDK_JAVA_OPTIONS`, `CLASSPATH`),
+  `GCONV_PATH`/`LOCPATH` and TLS trust stores (`SSL_CERT_FILE`, ...). Passing
+  one to an allow-listed program would let it load arbitrary code or config.
+  The grant is rejected at parse time on the interpreter and `-run-bc`;
+  generated Go and JavaScript skip such names when building the child
+  environment. An exec child that needs a search path should be granted by
+  absolute path; it starts with no `PATH`.
 * Receipt `grant` strings are copied verbatim from the normalized grant. An
   allowed exec records the matched entry with every pattern literal exactly
   as written (for example `process:exec=/usr/bin/git log --oneline *`).