diff --git a/apps/webhook_relay/README.md b/apps/webhook_relay/README.md index b6ea7497..b1c720f2 100644 --- a/apps/webhook_relay/README.md +++ b/apps/webhook_relay/README.md @@ -11,3 +11,12 @@ go run ../../howlframe.go -run-bc -allow-caps network,environment,network:fetch= ``` The `/relay` route is granted exactly one upstream, `network:fetch=127.0.0.1:8099` (#165). To relay elsewhere, change that route grant and pass the same grant to `-allow-caps`. + + +`UPSTREAM_URL` is read at request time; it does **not** change the route's +pinned authority. Only `127.0.0.1:8099` is granted, so setting the variable +to another hostname (including `localhost`) or port causes a capability +denial. A different upstream requires updating the route grant, the +runner's `-allow-caps`, and `UPSTREAM_URL` together. The literal grant is an +explicit opt-out from the default private/loopback DNS guard. Any path on +that origin, including `/admin`, is within this grant. diff --git a/change_log.md b/change_log.md index c4cc7894..090fa961 100644 --- a/change_log.md +++ b/change_log.md @@ -3,6 +3,7 @@ ## Unreleased ### Breaking +* Network destination hardening (#171, #172). On the capability client, generated Go and Node JavaScript, hostname grants now refuse DNS results in loopback (127/8, ::1), RFC1918 (10/8, 172.16/12, 192.168/16), ULA (fc00::/7), NAT64 (64:ff9b::/96), link-local and unspecified ranges at connection time. Explicit IP-literal grants and the exact name `localhost` opt out; allowed bytecode receipt effects record `private_destination_opt_out: "explicit IP-literal or localhost grant"`. Multicast (224/4, ff00::/8) is always refused, including explicit literals. Review follow-up (#176, part of #177): the blocked set is one shared table (`capability.SpecialPrefixes`) rendered into generated Go and JavaScript. It adds CGNAT 100.64/10 (including 100.100.100.200 metadata), 0/8, 192.0.0/24, 198.18/15, 240/4 (including broadcast), the documentation ranges, NAT64 64:ff9b:1::/48, Teredo 2001::/32, 100::/64, fec0::/10 and any non-global-unicast address. IPv4 inside IPv4-compatible (::a.b.c.d) and 6to4 (2002::/16) addresses is decoded and checked. A redirect may land on an IP-literal or `localhost` destination only when the request started on that same host: a hostname grant can no longer bounce into a granted private IP or localhost, and localhost cannot bounce into 127.0.0.1. HTTPS-to-HTTP redirects are refused even when both destinations are granted; Authorization and Cookie are stripped after any host, port or scheme change for the rest of the redirect chain. Ten-hop redirect caps and ignored HTTP_PROXY/HTTPS_PROXY are regression-tested on core, generated Go and JS; Node uses native HTTP requests with the checked DNS address pinned and no proxy agent, including under Node 24's NODE_USE_ENV_PROXY. JavaScript previously accepted short IPv4 grants such as `network:fetch=127.1` through WHATWG canonicalization; it now refuses ambiguous numeric grants and URLs like Go. Trailing-dot names report an invalid hostname rather than an ambiguous numeric host. Browser JavaScript retains host/redirect checks but cannot inspect DNS results. Migration: use an explicit IP-literal or `localhost:` grant only when private access is intended; granting a route its own localhost port also grants access to its `/admin`. `process:env=` now also refuses PYTHONPATH, PYTHONSTARTUP, NODE_OPTIONS, NODE_PATH, BASH_ENV, ENV, PERL5LIB, PERL5OPT, RUBYOPT and RUBYLIB, case-insensitively, alongside LD_*/DYLD_*; generated children omit them even if a raw grant bypasses parsing. `apps/webhook_relay` still reads UPSTREAM_URL at runtime but its route remains pinned to 127.0.0.1:8099; changing the environment does not expand its authority. No production HFIR flip; #90 stays Partial; pins, freeze tests and target exclusions unchanged. Reference: `docs/reference/network_authority.md`; evidence: `docs/journals/2026-10-13_network_destination_hardening.md`. * HFIR model adapter resource bounds (#169, #170): shared closed scalar `const`/`binary` subexpressions now lower once into private functions instead of exponentially expanded fragments; effects, variable reads and allocating expressions retain their evaluation sites and order. Candidates exceeding 2048 dependency levels, 65536 expanded nodes or 4194304 aggregate cached-fragment nodes fail closed with `HFIR_TRANSPORT_TOO_DEEP` / `HFIR_TRANSPORT_TOO_LARGE`, including callers that construct `Candidate` directly. Breaking for adapter consumers: in-memory programs returned by `CompileCandidate` now have a maximum one-second wall-clock budget (a shorter runner deadline still applies); a sleep beyond the remaining budget is refused immediately with `HFIR_DEADLINE_EXCEEDED`, which `try_let` cannot catch. Receipt paths report both the effective budget and typed failure. The adapter constraint is runner-owned and ephemeral; HFBC serialization remains unchanged, and artifact runners still select their execution policy. Explicit-deadline standalone VM sleeps also refuse durations beyond their remaining budget with the existing `LIMIT_EXCEEDED` code. Production HFIR compilation, #90 (Partial), target fences, SHA-256 pins, freeze tests and the Assurance tip-lock remain unchanged. Journal: [HFIR adapter resource blockers](docs/journals/2026-10-13_hfir_adapter_resource_blockers.md). * HFIR model adapter (#88) now covers the full `-compile-hfir-bc` language. `hfir-model-adapter/v1` accepts every node kind `LowerToBytecode` lowers from source (`hfir.AdapterNodeKinds`): `defun`/`param`/`call`/`return`, `while`, `for`, `try`/`catch`, `read_file`, `write_file`, `mkdir`, `exec`, `fetch`, `parse_json`, `is_nil`, `map_keys`, `html_escape`, `attr_escape`, `regex_match`, `time_now`, `sleep` (transport role `duration`), `cli_args`, `req_query`/`req_header`/`req_path`, and `(map_get key)` via a `target` edge, on top of the Phase-1 kinds. `for`/`while` control edges are derived from validated roles like `if`; the schema still has no control-edge field. Kinds the source lowerer does not lower (`spawn`, `spawn_agent`, `grants`, `llm_generate`, `http_server`/`res*`, store/db forms, `read_line`, `http_request`, `match`, ...) still fail with `HFIR_TRANSPORT_KIND`. No new opcode, capability, grant, lowering or language form; effects reached through the adapter are checked by the same run-time grants and fail closed without them. Breaking for adapter clients: candidate bounds rise to 4096 nodes, 64 inputs per node and 1 MiB per transport (from 128 / 32 / 64 KiB); the transport type check now types BOOL consts as bool, `to_int`/`to_float` as numbers and `str_split` as a list, and accepts string `+` string, so a candidate mixing a bool into arithmetic, or a `sleep` whose duration is a known non-number, is now rejected with `HFIR_TRANSPORT_TYPE`; a `return` outside a `defun` body is rejected with `HFIR_TRANSPORT_RETURN` unless it is a straight-line main-level statement (program/sequence/let body or if branch), which source already allows; cycles are rejected (`HFIR_TRANSPORT_CYCLE`) before type inference; non-const nodes never carry `literal_kind`. New experimental flag `-hfir-adapter-roundtrip` (with `-compile-hfir-bc`) sends the source graph through the adapter (encode, strict decode, `CompileCandidate`). `TestModelAdapterCoversArityTable` requires every one of the 386 corpus programs `-compile-hfir-bc` lowers to round-trip byte for byte, and every arity-table form to be covered (126 of 159) or carry an exemption with a rejected sample (33) that the test checks stays outside `-compile-hfir-bc`. The prod-flip criteria's model-adapter row is updated; the promote-blocker fence, production `-compile-bc`, #90 (Partial) and the tip-lock are unchanged. Pinned token benchmarks unchanged. * Network scoping (#165, #166). HowlFrame is a capability-governed runtime, not a sandbox; see `docs/reference/network_authority.md`. (1) The coarse `network` grant no longer authorizes outbound `fetch` / `http_request` on any path (interpreter, `-run-bc` for AST and HFIR artifacts, generated Go and JavaScript). Each host needs `network:fetch=[:]`; matching is exact (no wildcard or suffix match, IP literals only match the same canonical IP, ambiguous numeric hosts such as `127.1` are refused, an omitted port means the scheme default). Every redirect hop is re-checked, a granted DNS name may not connect to a link-local/unspecified address (cloud metadata), and proxy environment variables are ignored. Migration: replace `-allow-caps network` with `-allow-caps network,network:fetch=api.example.com` (keep `network` only if the program also serves HTTP or calls the model forms). (2) Route handlers no longer keep `network` implicitly: a route holds only its `(grants ...)` list or the server default. `res`, `res_json`, `res_header`, `http_res_header` and `req_method` no longer require any capability (`RES`, `RES_JSON`, `HTTP_RES_HEADER` and `HTTP_REQ_METHOD` now have an empty capability field). A route that fetches must list `network:fetch=` hosts, which the runner must hold (`apps/webhook_relay` now grants its upstream explicitly). (3) A denied fetch raises `CAPABILITY_DENIED` and the receipt's denied `FETCH`/`HTTP_REQUEST` effect carries `grant: "network:fetch=host:port"`; allowed fetches record the matched grant, and each redirect hop is its own effect. (4) Generated Go refuses `fetch`/`http_request` inside a route or middleware with `UNSUPPORTED_ON_TARGET`, because it cannot scope a single route. (5) A failing route handler now returns a generic `500 {"error":"internal server error"}` to the client; the runtime-error JSON is written to the server log (and receipt) only. (6) `process:env=` refuses dynamic-loader variables (`LD_*`, including `LD_PRELOAD`, and `DYLD_*`, case-insensitive) on every path, and generated Go/JS never pass them to exec children. (7) `-required-caps` reports `network:fetch=host:port` for constant-URL fetches and `network:fetch=` for computed URLs instead of `network`. Receipt `grant` strings are copied verbatim from the normalized grant, including `process:exec=` pattern literals; do not put secrets in grant patterns. diff --git a/docs/journals/2026-10-13_network_destination_hardening.md b/docs/journals/2026-10-13_network_destination_hardening.md new file mode 100644 index 00000000..8aecd01c --- /dev/null +++ b/docs/journals/2026-10-13_network_destination_hardening.md @@ -0,0 +1,172 @@ +# Network destination hardening (#171, #172) + +Date: 2026-10-13. Worktree: hf-blockers-net, based on main 1bda2f15. +No commit or push. + +## Consumer findings and changes + +The three network implementations needed the same destination policy. +Hostname grants now block loopback, RFC1918, ULA, NAT64, link-local, +unspecified and multicast results. Literal IP grants and the exact name +localhost deliberately permit private access; multicast is always refused. +Deterministic lookup overrides resolve private.test to 127.0.0.1 in the +socket tests: the target receives zero requests. Literal and localhost +opt-outs each reach the target. Boundary fixtures cover all requested +ranges, IPv4-mapped IPv6, and public addresses beside private ranges. + +Go checks every returned address before dialing, then checks the selected +address again with Dialer.Control. Node checks DNS results and pins its +native HTTP connection's lookup to a checked result. A rebinding fixture +returns a public address first and loopback on any second lookup; the +request connects using the first result and performs exactly one lookup. +Browser execution keeps host, literal multicast and redirect checks but +cannot inspect DNS or pin connections. No DOM was added. + +An 11-redirect chain stops after the initial request plus ten followed +redirects. Proxy tests count TCP connections, not just HTTP handler calls. +HTTP and HTTPS traps see zero connections in core and generated Go. +Node 24.21.0, selected with `npx --yes --package=node@24`, also passes with +HTTP_PROXY, HTTPS_PROXY and NODE_USE_ENV_PROXY=1 set at process startup. +Mutating back to native fetch produces a proxy connection and fails. +The startup trap closes connections immediately so a mutated CONNECT +request cannot stall the test. An earlier trap that served an ordinary +HTTP response stalled that mutation; it was replaced, the task's leftover +test processes were stopped, and the mutation was rerun successfully. +Node 20.19.2 is the default environment. + +HTTPS downgrades are refused before contacting the HTTP destination even +when both ports are granted. Port-changing redirects strip Authorization +and Cookie. A three-hop Go consumer revealed that net/http can copy the +initial request's headers again on a later same-origin hop; stripping now +continues for the rest of the chain after an origin change. Generated Go +also compares effective origins, preserving credentials across a change +in hostname case or spelling of the default port. Node's native response +reader retains gzip/deflate/Brotli decoding and UTF-8 BOM removal from the +previous fetch behavior. + +Allowed bytecode fetch receipt effects record +`private_destination_opt_out` for literal or localhost authority, including +redirect effects. This records permission, not successful network access. +Denied effects and ordinary hostname grants do not record that opt-out. + +Process grants refuse interpreter code-loading variables case-insensitively: +PYTHONPATH, PYTHONSTARTUP, NODE_OPTIONS, NODE_PATH, BASH_ENV, ENV, PERL5LIB, +PERL5OPT, RUBYOPT and RUBYLIB, alongside existing LD_*/DYLD_* refusals. +Generated Go/JS child-environment tests also supply raw grants directly, +so bypassing grant parsing does not pass the variables through. + +WHATWG previously canonicalized JS grants such as 127.1 to 127.0.0.1. +JS now refuses ambiguous numeric grants and URL authorities; strict URL +syntax prevents alternate slash/backslash forms from bypassing the check. +Go trailing-dot hostname diagnostics now identify the trailing dot rather +than reporting an ambiguous numeric host. + +## Breaking changes and app documentation + +The Unreleased Breaking entry and network/process authority references +describe the tightened name-destination policy, multicast/downgrade +refusals, stripped credentials, receipt field, environment exclusions and +JS numeric-grant correction. Private access requires explicit literal or +localhost authority. Granting a route localhost: permits +its /admin and all other paths on that origin. + +apps/webhook_relay/README.md now makes the runtime UPSTREAM_URL selection +and fixed 127.0.0.1:8099 route authority explicit. Changing the variable +cannot expand the grant; changing upstream requires updating both route +and runner grants. The app source is unchanged, preserving token fixtures. + +## Negative tests and mutation evidence + +Each mutation below temporarily changed only the named production guard; +the selected test failed behaviorally (not because compilation failed). +The original source was restored in a finally block after every run. +All 34 mutations were detected. + +| Guard / compatibility behavior | Negative test | Result | +|---|---|---| +| core redirect cap | `TestFetchSecurity/RedirectCap11` | failed with mutation; restored | +| core proxy isolation | `TestFetchSecurity/ProxyEnvIgnored` | failed with mutation; restored | +| core HTTPS downgrade | `TestFetchSecurity/HTTPSDowngrade` | failed with mutation; restored | +| core multicast literals | `TestFetchSecurity/MulticastDial` | failed with mutation; restored | +| core private-name guard | `TestFetchSecurity/NamePrivateDial` | failed with mutation; restored | +| core private-range guard | `TestFetchPrivateRanges` | failed with mutation; restored | +| core credential stripping | `TestFetchSecurity/CredentialsPortChange` | failed with mutation; restored | +| core code-loading environment | `TestEnvGrantRefusesCodeLoading` | failed with mutation; restored | +| core trailing-dot diagnostic | `TestFetchTrailingDotDiagnostic` | failed with mutation; restored | +| receipt opt-out | `TestReceiptFetchPrivateOptOut` | failed with mutation; restored | +| gogen redirect cap | `TestGoFetchSecurity → TestSecurity/RedirectCap11` | failed with mutation; restored | +| gogen proxy isolation | `TestGoFetchSecurity → TestSecurity/ProxyEnvIgnored` | failed with mutation; restored | +| gogen HTTPS downgrade | `TestGoFetchSecurity → TestSecurity/HTTPSDowngrade` | failed with mutation; restored | +| gogen multicast literals | `TestGoFetchSecurity → TestSecurity/MulticastDial` | failed with mutation; restored | +| gogen private-name guard | `TestGoFetchSecurity → TestSecurity/NamePrivateDial` | failed with mutation; restored | +| gogen private-range guard | `TestGoFetchSecurity → TestSecurity/PrivateRangesDial` | failed with mutation; restored | +| gogen credential stripping | `TestGoFetchSecurity → TestSecurity/CredentialsPortChange` | failed with mutation; restored | +| gogen code-loading environment | `TestGoFetchSecurity → TestCodeLoadingEnv` | failed with mutation; restored | +| JS redirect cap | `TestJSRedirectCap11` | failed with mutation; restored | +| JS proxy isolation | `TestJSProxyEnvIgnored` | failed with mutation; restored | +| JS HTTPS downgrade | `TestJSHTTPSDowngrade` | failed with mutation; restored | +| JS multicast guard | `TestJSMulticastDial` | failed with mutation; restored | +| JS private-name guard | `TestJSNamePrivateDial` | failed with mutation; restored | +| JS private-range guard | `TestJSPrivateRanges` | failed with mutation; restored | +| JS credential stripping | `TestJSCredentialsPortChange` | failed with mutation; restored | +| JS numeric grants | `TestJSAmbiguousNumericGrant` | failed with mutation; restored | +| JS numeric URLs | `TestJSAmbiguousNumericGrant` | failed with mutation; restored | +| JS code-loading environment | `TestJSCodeLoadingEnv` | failed with mutation; restored | +| JS DNS pin | `TestJSPinsVettedLookup` | failed with mutation; restored | +| JS HTTPS downgrade real TLS | `TestJSHTTPSDowngrade` | failed with mutation; restored | +| Node 24 startup proxy isolation | `TestJSProxyEnvAtStartup (Node 24)` | failed with mutation; restored | +| JS strict numeric URL syntax | `TestJSAmbiguousNumericGrant` | failed with mutation; restored | +| JS response decoding | `TestJSFetchCompressedResponse` | failed with mutation; restored | +| gogen effective-origin compatibility | `TestGoFetchSecurity → TestSecurity/CredentialsSameOrigin` | failed with mutation; restored | + +Core socket assertions run under `TestFetchSecurity`. Generated Go emits +the authoritative fetch/process helper sources into a temporary consumer +package under internal/ and runs the same assertions under +`TestGoFetchSecurity` → `TestSecurity`; temporary artifacts are removed. +Mutation filtering uses HF_NET_MUTATION_FILTER for that inner consumer. +The test fixture formerly treating private IPs as public was replaced by +public IPv4/IPv6 fixtures and explicit private-range refusal tests. +The JS code-shape assertion now names the governed request helper; network +behavior is asserted separately. No freeze test or sha256 pin was changed. + +Mutation transcripts were kept outside the repository under +`/tmp/hf-net-work/mutation-*.log` during this session. The table above is +the durable record of the selected tests and restored guards. + +## Validation + +* Formatting and git diff whitespace checks passed. +* go build ./... and go vet ./... passed. +* go test -json ./... passed with zero failed test/package events. +* HFIR parity: 268 leaf cases passed, 0/268 mismatches. +* TestModelAdapterCoversArityTable and TestProdFlipCriteriaLock passed. +* TestHFIRParseJSONWriteFileStaysFailClosed, the governed-profile tests and + TestIssue165GoRouteFetchFailsClosed passed; Go/JS UNSUPPORTED_ON_TARGET + behavior remains intact. +* Existing frozen-result/pin suites passed. No sha256 pin, freeze test, + production HFIR selection, #90 Partial status or tip-lock was changed. +* Node 24.21.0 passed private DNS, pinned lookup, downgrade, port-changing + credential, proxy-at-startup and compressed-response consumers after all + mutations were restored. +* Final go test ./...: pending the last run after this journal update. + +## Files changed + +* `apps/webhook_relay/README.md` +* `change_log.md` +* `docs/journals/2026-10-13_network_destination_hardening.md` +* `docs/reference/network_authority.md` +* `docs/reference/process_authority.md` +* `internal/backend/gogen/gogen.go` +* `internal/backend/gogen/network_security_test.go` +* `internal/backend/javascript/javascript.go` +* `internal/backend/javascript/javascript_test.go` +* `internal/backend/javascript/network_security_test.go` +* `internal/capability/grants.go` +* `internal/capability/network.go` +* `internal/capability/network_security_test.go` +* `internal/capability/network_test.go` +* `internal/capability/process.go` +* `internal/nettest/security.go` +* `internal/vm/receipt.go` +* `internal/vm/receipt_test.go` diff --git a/docs/reference/network_authority.md b/docs/reference/network_authority.md index a5ba1bea..fea92eaf 100644 --- a/docs/reference/network_authority.md +++ b/docs/reference/network_authority.md @@ -1,4 +1,4 @@ -# Network authority (#165) +# Network authority (#165, #171, #172) HowlFrame is a capability-governed runtime, not a sandbox. Grants are checked inside the host process before an effect runs; they are not an @@ -40,11 +40,40 @@ capability at all: they only touch the request the server already accepted. * Every redirect hop is re-checked against the same allow-list (at most 10 hops). A hop to an ungranted host stops the fetch with `CAPABILITY_DENIED` before that host is contacted. -* A granted DNS name may not connect to a link-local, multicast or - unspecified address (`169.254.0.0/16`, `fe80::/10`, `0.0.0.0`, `::`), which - is where cloud metadata services live. Granting the IP literal itself (for - example `network:fetch=169.254.169.254`) is the only way to reach one. -* `HTTP_PROXY` / `HTTPS_PROXY` are ignored by governed fetches. +* Hostname grants block DNS results in loopback (`127.0.0.0/8`, `::1`), + RFC1918 (`10/8`, `172.16/12`, `192.168/16`), IPv6 ULA (`fc00::/7`), + NAT64 (`64:ff9b::/96`), link-local (`169.254/16`, `fe80::/10`) and + unspecified ranges, plus every other non-global-unicast or special-purpose + range in the shared table `capability.SpecialPrefixes`: CGNAT + (`100.64/10`, which includes the `100.100.100.200` metadata address), + `0/8`, `192.0.0/24`, `198.18/15`, `240/4` (including broadcast), the + documentation ranges, `64:ff9b:1::/48`, Teredo (`2001::/32`), `100::/64` + and `fec0::/10`. Generated Go and JavaScript render the same table. + IPv4-mapped IPv6 is checked as IPv4, and IPv4 embedded in IPv4-compatible + (`::a.b.c.d`) and 6to4 (`2002::/16`) addresses is decoded and checked. Go checks the + resolved address before dialing and again at socket creation. Node pins + its connection to the address checked by its lookup. +* Explicit IP-literal grants and the exact name `localhost` are deliberate + opt-outs for those ranges. They still require the exact host and port; + `localhost` does not authorize `127.0.0.1`, or vice versa. Multicast + (`224.0.0.0/4`, `ff00::/8`) is always refused, even with a literal grant. +* A redirect may land on an IP-literal or `localhost` destination only if + the request started on that same host. A hostname grant can't be + redirected into a granted private IP or `localhost`, and `localhost` + can't be redirected to `127.0.0.1` (or the reverse). +* HTTPS-to-HTTP redirects are refused even when both hosts are granted. + On any host, effective port or scheme change, Authorization and Cookie + headers are removed for the rest of the chain, including subsequent hops + on the new origin. +* `HTTP_PROXY` / `HTTPS_PROXY` (and lowercase variants) are ignored by + governed fetches. Node 24 with `NODE_USE_ENV_PROXY=1` honors proxy env in + its native fetch; the governed Node path uses native HTTP requests with + `agent: false` and a pinned lookup instead. The proxy trap test also runs + with that setting enabled at Node startup. +* JavaScript previously accepted `network:fetch=127.1` because WHATWG URL + parsing canonicalized it to `127.0.0.1`. Ambiguous numeric grants and URL + hosts are now refused like Go. Trailing-dot names are invalid hostnames. + ## Denials and receipts @@ -58,7 +87,11 @@ On `-run-bc` the receipt records a denied `FETCH` / `HTTP_REQUEST` effect whose `grant` field is the grant that would have authorized it (`network:fetch=host:port`; never the path, query or userinfo). An allowed fetch records the matched grant, and each followed redirect hop is recorded -as its own effect with target summary `redirect`. +as its own effect with target summary `redirect`. An allowed effect under +an explicit IP-literal or localhost grant also records +`private_destination_opt_out: "explicit IP-literal or localhost grant"`. +This reports permission to reach private destinations; it does not claim +that the request successfully connected or reached a private IP. Receipt `grant` strings are copied verbatim from the normalized grant. For `process:exec=` entries that includes every pattern literal exactly as @@ -79,6 +112,16 @@ host: (lambda (req) (http_res (http_request "https://api.example.com/v1" "GET"))))) ``` +Granting `network:fetch=localhost:` lets a route reach its own +`/admin` (and every other path on that origin). Grants do not restrict URL +paths. An explicit loopback literal grants equivalent access when used +as the URL host. + +The `apps/webhook_relay` route is pinned to `127.0.0.1:8099` but reads +`UPSTREAM_URL` at request time. That environment variable selects the URL, +not the grant: any other host or port is denied. To change the upstream, +update the route grant and the runner's grant as well as `UPSTREAM_URL`. + The runner must hold every grant a route lists, or registration fails with `CAPABILITY_DENIED` at `HTTP_ROUTE` before the server listens. @@ -94,8 +137,8 @@ recorded). | Interpreter (`-run`) | Same host matching, redirect re-check and dial guard. No receipts. | | `-run-bc` (AST and HFIR artifacts) | Full enforcement plus receipts. | | Generated Go | Same host matching, redirect re-check and dial guard from `HOWLFRAME_ALLOW_CAPS`. Generated Go runs every route with the whole process's grants, so `fetch` / `http_request` inside a route or middleware fails closed at build time with `UNSUPPORTED_ON_TARGET`. | -| Generated JavaScript (Node) | Same host matching (WHATWG-canonical hosts) and per-hop redirect re-check (`redirect: "manual"`); DNS names are resolved first and refused if they resolve to a link-local or unspecified address (a resolve-then-connect check, so a rebinding race remains). | -| Generated JavaScript (browser) | Same host matching. Opaque redirects are refused. A browser cannot resolve names, so the link-local guard does not apply there. | +| Generated JavaScript (Node) | Same strict grant matching, per-hop redirect checks and private/multicast guards; checked DNS addresses are pinned into native HTTP requests, which ignore proxy env. No receipt emission on this target. | +| Generated JavaScript (browser) | Same host matching. Opaque redirects are refused. Literal multicast and HTTPS downgrades are refused. A browser cannot inspect DNS results or pin a connection, so private-name DNS enforcement depends on browser/network policy; it cannot provide the Node dial guard. | ## Required-capabilities profile diff --git a/docs/reference/process_authority.md b/docs/reference/process_authority.md index bd06c9e7..62f06c3e 100644 --- a/docs/reference/process_authority.md +++ b/docs/reference/process_authority.md @@ -120,7 +120,9 @@ Rules, enforced identically on the interpreter (`-run`), `-run-bc` (both * `process:env=NAME` refuses dynamic-loader variables: any name starting with `LD_` (including `LD_PRELOAD`, `LD_LIBRARY_PATH`, `LD_AUDIT`) or `DYLD_`, - compared case-insensitively. Passing one to an allow-listed program would + plus interpreter code-loading variables `PYTHONPATH`, `PYTHONSTARTUP`, + `NODE_OPTIONS`, `NODE_PATH`, `BASH_ENV`, `ENV`, `PERL5LIB`, `PERL5OPT`, + `RUBYOPT` and `RUBYLIB`, compared case-insensitively. Passing one to an allow-listed program would let it load arbitrary code. The grant is rejected at parse time on the interpreter and `-run-bc`; generated Go and JavaScript skip such names when building the child environment. diff --git a/internal/backend/gogen/gogen.go b/internal/backend/gogen/gogen.go index 1a0e9ae2..e8c44fd8 100644 --- a/internal/backend/gogen/gogen.go +++ b/internal/backend/gogen/gogen.go @@ -3,6 +3,7 @@ package gogen import ( "fmt" "github.com/howlcipher/howlframe/internal/ast" + "github.com/howlcipher/howlframe/internal/capability" "github.com/howlcipher/howlframe/internal/httpreq" "github.com/howlcipher/howlframe/internal/ir" "strconv" @@ -589,11 +590,19 @@ func howlFrameExecSlot(p, arg string) bool { } // howlFrameExecEnv passes only process:env=NAME variables; never the host env. +func howlFrameCodeLoadingEnv(name string) bool { + switch strings.ToUpper(name) { + case "PYTHONPATH", "NODE_OPTIONS", "BASH_ENV", "ENV", "PERL5LIB", "PERL5OPT", "RUBYOPT", "RUBYLIB", "PYTHONSTARTUP", "NODE_PATH": + return true + } + return false +} + func howlFrameExecEnv() []string { env := []string{} for _, grant := range strings.Split(os.Getenv("HOWLFRAME_ALLOW_CAPS"), ",") { name, ok := strings.CutPrefix(strings.TrimSpace(grant), "process:env=") - if !ok || name == "" || strings.ContainsAny(name, "= ") || strings.HasPrefix(strings.ToUpper(name), "LD_") || strings.HasPrefix(strings.ToUpper(name), "DYLD_") { + if !ok || name == "" || strings.ContainsAny(name, "= ") || strings.HasPrefix(strings.ToUpper(name), "LD_") || strings.HasPrefix(strings.ToUpper(name), "DYLD_") || howlFrameCodeLoadingEnv(name) { continue } if v, ok := os.LookupEnv(name); ok { @@ -615,7 +624,6 @@ func howlFrameExec(cmd string, args ...string) []byte { } return out } - ` } @@ -668,12 +676,64 @@ func fetchImports(extra []string) string { return out } +// specialPrefixGoSource renders capability.SpecialPrefixes, the one table +// shared by every target, into the emitted helper. +func specialPrefixGoSource() string { + var b strings.Builder + b.WriteString("var howlFrameSpecialPrefixes = []netip.Prefix{\n") + for _, p := range capability.SpecialPrefixes { + fmt.Fprintf(&b, "\tnetip.MustParsePrefix(%q),\n", p) + } + b.WriteString("}\n\n") + b.WriteString(`// howlFrameForbiddenDest mirrors capability.ForbiddenDestination. +func howlFrameForbiddenDest(a netip.Addr) bool { + a = a.Unmap() + if a.Is6() { + b := a.As16() + if b[0] == 0x20 && b[1] == 0x02 && howlFrameForbiddenDest(netip.AddrFrom4([4]byte{b[2], b[3], b[4], b[5]})) { + return true + } + compat := true + for _, x := range b[:12] { + compat = compat && x == 0 + } + if compat && !(b[12] == 0 && b[13] == 0 && b[14] == 0) { + return howlFrameForbiddenDest(netip.AddrFrom4([4]byte{b[12], b[13], b[14], b[15]})) + } + } + if !a.IsGlobalUnicast() { + return true + } + for _, p := range howlFrameSpecialPrefixes { + if p.Contains(a) { + return true + } + } + return false +} + +// howlFrameOptOutHost reports an IP-literal or localhost destination. +func howlFrameOptOutHost(u *url.URL) (string, bool) { + h, _ := howlFrameFetchSplit(u.Host) + n, ok := howlFrameFetchNormHost(h) + if !ok { + return "", false + } + _, err := netip.ParseAddr(n) + return n, err == nil || n == "localhost" +} + +`) + return b.String() +} + func fetchHelperSource() string { - return `// howlFrameFetchAllowed mirrors capability.Grants.AuthorizeFetch: the URL's + return specialPrefixGoSource() + `// howlFrameFetchAllowed mirrors capability.Grants.AuthorizeFetch: the URL's // host must equal a network:fetch=host[:port] grant exactly (no suffix // match; an omitted port means the scheme default). The coarse network grant // never authorizes an outbound request. Redirect hops are re-checked and a -// granted DNS name may not connect to a link-local or unspecified address. +// granted DNS name may not connect to a private or metadata address unless +// it is localhost. Multicast and HTTPS downgrades are always refused. func howlFrameFetchNormHost(h string) (string, bool) { if strings.HasPrefix(h, "[") { if !strings.HasSuffix(h, "]") { @@ -773,6 +833,27 @@ type howlFrameFetchDenied struct{} func (howlFrameFetchDenied) Error() string { return "capability denied: network" } +func howlFrameFetchSameOrigin(a, b *url.URL) bool { + origin := func(u *url.URL) string { + h, p := howlFrameFetchSplit(u.Host) + h, ok := howlFrameFetchNormHost(h) + if !ok { + return "" + } + scheme := strings.ToLower(u.Scheme) + if p == "" { + p = "80" + if scheme == "https" { + p = "443" + } + } + return scheme + "://" + net.JoinHostPort(h, p) + } + return origin(a) != "" && origin(a) == origin(b) +} + +var howlFrameFetchLookupIP = net.DefaultResolver.LookupIPAddr + var howlFrameFetchClient = &http.Client{ Transport: &http.Transport{ Proxy: nil, @@ -788,15 +869,56 @@ var howlFrameFetchClient = &http.Client{ return howlFrameFetchDenied{} } a := ap.Addr().Unmap() - if litErr != nil && (a.IsLinkLocalUnicast() || a.IsLinkLocalMulticast() || a.IsUnspecified() || a.IsMulticast()) { + if a.IsMulticast() || litErr != nil && !strings.EqualFold(host, "localhost") && howlFrameForbiddenDest(a) { return howlFrameFetchDenied{} } return nil }} + if litErr != nil { + _, port, _ := net.SplitHostPort(addr) + addresses, err := howlFrameFetchLookupIP(ctx, host) + if err != nil { + return nil, err + } + for _, address := range addresses { + if err := d.Control("", net.JoinHostPort(address.IP.String(), port), nil); err != nil { + return nil, err + } + } + var last error + for _, address := range addresses { + conn, err := d.DialContext(ctx, network, net.JoinHostPort(address.IP.String(), port)) + if err == nil { + return conn, nil + } + last = err + } + if last != nil { + return nil, last + } + return nil, howlFrameFetchDenied{} + } return d.DialContext(ctx, network, addr) }, }, CheckRedirect: func(req *http.Request, via []*http.Request) error { + previous := via[len(via)-1].URL + if previous.Scheme == "https" && req.URL.Scheme == "http" { + return howlFrameFetchDenied{} + } + if host, opt := howlFrameOptOutHost(req.URL); opt { + if start, _ := howlFrameOptOutHost(via[0].URL); start != host { + return howlFrameFetchDenied{} + } + } + changedOrigin := !howlFrameFetchSameOrigin(previous, req.URL) + for i := 1; i < len(via); i++ { + changedOrigin = changedOrigin || !howlFrameFetchSameOrigin(via[i-1].URL, via[i].URL) + } + if changedOrigin { + req.Header.Del("Authorization") + req.Header.Del("Cookie") + } if len(via) > 10 || !howlFrameFetchAllowed(req.URL) { return howlFrameFetchDenied{} } @@ -832,7 +954,6 @@ func howlFrameFetchBytes(url string, method string, body io.Reader) []byte { } return b } - ` } diff --git a/internal/backend/gogen/network_security_test.go b/internal/backend/gogen/network_security_test.go new file mode 100644 index 00000000..3c248c67 --- /dev/null +++ b/internal/backend/gogen/network_security_test.go @@ -0,0 +1,59 @@ +package gogen + +import ( + "os" + "os/exec" + "path/filepath" + "testing" +) + +func TestGoFetchSecurity(t *testing.T) { + // Compile the exact helper emitted by the authoritative generator into a + // temporary consumer package, and run the same socket tests as core. + dir, err := os.MkdirTemp(filepath.Join(moduleRoot(t), "internal"), ".netscope-") + if err != nil { + t.Fatal(err) + } + defer os.RemoveAll(dir) + source := `package consumer +import ( + "context"; "errors"; "fmt"; "io"; "net"; "net/http"; "net/netip"; "net/url"; "os"; "os/exec"; "strconv"; "strings"; "syscall"; "time"; "testing" + "github.com/howlcipher/howlframe/internal/nettest" +) +` + fetchHelperSource() + execHelperSource() + ` +func TestSecurity(t *testing.T) { + nettest.Run(t,func(grants ...string)*http.Client { + t.Setenv("HOWLFRAME_ALLOW_CAPS",strings.Join(grants,",")) + c:=*howlFrameFetchClient + c.Transport=howlFrameFetchClient.Transport.(*http.Transport).Clone() + return &c + },func(f nettest.Lookup)func(){ + old:=howlFrameFetchLookupIP;howlFrameFetchLookupIP=f;return func(){howlFrameFetchLookupIP=old} + }) +} +func TestCodeLoadingEnv(t *testing.T) { + for _,name:=range []string{"PYTHONPATH","NODE_OPTIONS","BASH_ENV","ENV","PERL5LIB","PERL5OPT","RUBYOPT","RUBYLIB","PYTHONSTARTUP","NODE_PATH"} { + for _,raw:=range []string{name,strings.ToLower(name)} { + t.Setenv(raw,"injected") + t.Setenv("HOWLFRAME_ALLOW_CAPS","process:env="+raw) + if env:=howlFrameExecEnv();len(env)!=0 {t.Fatalf("code-loading variable passed: %v",env)} + } + } +} +` + if err := os.WriteFile(filepath.Join(dir, "security_test.go"), []byte(source), 0600); err != nil { + t.Fatal(err) + } + args := []string{"test", "-count=1", "-v", dir} + // Optional filter is used by the mutation runner to select one assertion. + if filter := os.Getenv("HF_NET_MUTATION_FILTER"); filter != "" { + args = append(args, "-run", filter) + } + cmd := exec.Command("go", args...) + cmd.Dir = moduleRoot(t) + out, err := cmd.CombinedOutput() + if err != nil { + t.Fatalf("emitted helper conformance: %v\n%s", err, out) + } + t.Log(string(out)) +} diff --git a/internal/backend/javascript/javascript.go b/internal/backend/javascript/javascript.go index 52402bdf..5ba6ff1a 100644 --- a/internal/backend/javascript/javascript.go +++ b/internal/backend/javascript/javascript.go @@ -3,7 +3,10 @@ package javascript import ( "fmt" "github.com/howlcipher/howlframe/internal/ast" + "github.com/howlcipher/howlframe/internal/capability" "github.com/howlcipher/howlframe/internal/ir" + "net/netip" + "strconv" "strings" ) @@ -371,7 +374,7 @@ function howlFrameExecEnv() { var grant = grants[i].trim(); if (grant.indexOf("process:env=") !== 0) continue; var name = grant.slice("process:env=".length); - if (name === "" || /[= ]/.test(name) || /^(LD_|DYLD_)/i.test(name)) continue; + if (name === "" || /[= ]/.test(name) || /^(LD_|DYLD_)/i.test(name) || /^(PYTHONPATH|NODE_OPTIONS|BASH_ENV|ENV|PERL5LIB|PERL5OPT|RUBYOPT|RUBYLIB|PYTHONSTARTUP|NODE_PATH)$/i.test(name)) continue; if (Object.prototype.hasOwnProperty.call(process.env, name)) env[name] = process.env[name]; } return env; @@ -420,7 +423,7 @@ func readFileJSHelper() string { // which bytes_to_string prints unchanged. A third argument is the body; // a two-argument call sends none. func fetchJSHelper() string { - return `function howlFrameGrantList() { + return specialPrefixJSSource() + `function howlFrameGrantList() { var raw = ""; if (typeof process !== "undefined" && process.env) { raw = process.env.HOWLFRAME_ALLOW_CAPS || ""; @@ -446,6 +449,8 @@ function howlFrameFetchAllowed(u) { if (spec === "" || /[\/@?#*%\s\\]/.test(spec) || spec.charAt(spec.length - 1) === ":") continue; var g; try { g = new URL("http://" + spec); } catch (e) { continue; } + var writtenHost = spec.charAt(0) === "[" ? spec.slice(0, spec.indexOf("]") + 1) : spec.split(":")[0].toLowerCase(); + if (writtenHost.charAt(0) !== "[" && /(^|\.)[0-9][^.]*$/.test(writtenHost) && writtenHost !== g.hostname) continue; if (g.hostname !== u.hostname) continue; var gport = /:\d+$/.test(spec) ? spec.slice(spec.lastIndexOf(":") + 1) : def; if (String(Number(gport)) === gport && gport === port) return true; @@ -453,42 +458,157 @@ function howlFrameFetchAllowed(u) { return false; } +function howlFrameFetchURL(raw, base) { + var written = String(raw); + if (written.indexOf("\\") >= 0 || /^https?:/i.test(written) && !/^https?:\/\/[^/]/i.test(written)) howlFrameFetchDeny(); + var u = new URL(written, base); + // WHATWG rewrites short, octal and hexadecimal IPv4 before grant matching. + var authority = /^(?:[a-z][a-z0-9+.-]*:)?\/\/([^/?#]*)/i.exec(written); + if (authority) { + var host = authority[1].slice(authority[1].lastIndexOf("@") + 1).split(":")[0].toLowerCase(); + if (host.charAt(0) !== "[" && /(^|\.)[0-9][^.]*$/.test(host) && host !== u.hostname) howlFrameFetchDeny(); + } + return u; +} + function howlFrameFetchDeny() { throw new Error("CAPABILITY_DENIED: capability denied: network"); } -// A granted DNS name may not resolve to a link-local or unspecified address -// (cloud metadata). Node checks this before connecting; a browser cannot -// resolve names, so there the browser's own network policy applies. +// Node checks private and metadata DNS results and pins the connection. A +// browser cannot inspect DNS, so its own network policy governs named hosts. +function howlFrameAddrBytes(a) { + a = String(a).toLowerCase().replace(/^\[|\]$/g, ""); + var v4 = /^(\d+)\.(\d+)\.(\d+)\.(\d+)$/.exec(a); + if (v4) { + var out4 = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 255, 255]; + for (var k = 1; k <= 4; k++) { var n4 = Number(v4[k]); if (n4 > 255) return null; out4.push(n4); } + return out4; + } + if (a.indexOf(":") < 0 || a.indexOf("%") >= 0) return null; + var tail = /^(.*:)(\d+\.\d+\.\d+\.\d+)$/.exec(a); + if (tail) { + var t = howlFrameAddrBytes(tail[2]); + if (!t) return null; + a = tail[1] + ((t[12] << 8) | t[13]).toString(16) + ":" + ((t[14] << 8) | t[15]).toString(16); + } + var halves = a.split("::"); + if (halves.length > 2) return null; + var head = halves[0] ? halves[0].split(":") : []; + var rest = halves.length === 2 && halves[1] ? halves[1].split(":") : []; + var fill = 8 - head.length - rest.length; + if (halves.length === 1 ? fill !== 0 : fill < 1) return null; + var groups = head.concat(new Array(halves.length === 2 ? fill : 0).fill("0"), rest); + var out = []; + for (var g = 0; g < groups.length; g++) { + if (!/^[0-9a-f]{1,4}$/.test(groups[g])) return null; + var n = parseInt(groups[g], 16); + out.push(n >> 8, n & 255); + } + return out; +} + +// howlFrameIsSpecial mirrors capability.ForbiddenDestination over the shared table. +function howlFrameIsSpecial(b) { + var mapped = function (x) { return [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 255, 255].concat(x); }; + if (b[0] === 0x20 && b[1] === 0x02 && howlFrameIsSpecial(mapped(b.slice(2, 6)))) return true; + var compat = b.slice(0, 12).every(function (x) { return x === 0; }); + if (compat && !(b[12] === 0 && b[13] === 0 && b[14] === 0)) return howlFrameIsSpecial(mapped(b.slice(12, 16))); + for (var i = 0; i < howlFrameSpecialPrefixes.length; i++) { + var p = howlFrameSpecialPrefixes[i][0], bits = howlFrameSpecialPrefixes[i][1], ok = true; + for (var k = 0; k < 16 && ok && bits > 0; k++, bits -= 8) { + var mask = bits >= 8 ? 255 : (0xff << (8 - bits)) & 255; + ok = (b[k] & mask) === (p[k] & mask); + } + if (ok) return true; + } + return false; +} + +function howlFrameForbiddenAddress(address, explicit) { + var b = howlFrameAddrBytes(address); + if (!b) return true; + var v4 = b.slice(0, 12).join(",") === "0,0,0,0,0,0,0,0,0,0,255,255"; + if (v4 && b[12] >= 224 && b[12] <= 239 || b[0] === 0xff) return true; + if (explicit) return false; + return howlFrameIsSpecial(b); +} + async function howlFrameFetchResolvedOk(u) { - var host = u.hostname; - if (/^\[.*\]$/.test(host) || /^\d+\.\d+\.\d+\.\d+$/.test(host)) return true; + var host = u.hostname.replace(/^\[|\]$/g, ""); + var literal = /^\[.*\]$/.test(u.hostname) || /^\d+\.\d+\.\d+\.\d+$/.test(host); + if (literal) return !howlFrameForbiddenAddress(host, true); if (typeof require !== "function") return true; var addrs = await require("dns").promises.lookup(host, { all: true }); + if (!addrs.length) return false; for (var i = 0; i < addrs.length; i++) { - var a = String(addrs[i].address).toLowerCase(); - if (/^169\.254\./.test(a) || /^fe[89ab][0-9a-f]:/.test(a) || a === "0.0.0.0" || a === "::" || /^::ffff:169\.254\./.test(a)) return false; + if (howlFrameForbiddenAddress(addrs[i].address, host === "localhost")) return false; } + // Pin this request to a vetted address; do not resolve again at connect time. + u.howlFrameAddress = addrs[0]; return true; } -async function howlFrameFetch(url, method, body) { +function howlFrameFetchRequest(u, init) { + if (typeof require !== "function") return fetch(u.href, init); + // Native requests ignore proxy environment variables, including Node 24's + // NODE_USE_ENV_PROXY. agent:false also prevents pooled connections bypassing lookup. + return new Promise(function (resolve, reject) { + var options = { method: init.method, headers: init.headers, agent: false }; + if (u.howlFrameAddress) options.lookup = function (host, opts, cb) { + var a = u.howlFrameAddress; + cb(null, opts && opts.all ? [{ address: a.address, family: a.family }] : a.address, a.family); + }; + var req = require(u.protocol === "https:" ? "https" : "http").request(u, options, function (res) { + var chunks = []; + res.on("data", function (chunk) { chunks.push(chunk); }); + res.on("error", reject); + res.on("end", function () { resolve({ + status: res.statusCode, + headers: { get: function (name) { return res.headers[name.toLowerCase()]; } }, + text: async function () { + var data = Buffer.concat(chunks); + var encoding = String(res.headers["content-encoding"] || "").toLowerCase(); + if (encoding === "gzip") data = require("zlib").gunzipSync(data); + else if (encoding === "deflate") data = require("zlib").inflateSync(data); + else if (encoding === "br") data = require("zlib").brotliDecompressSync(data); + return data.toString("utf8").replace(/^\uFEFF/, ""); + } + }); }); + }); + req.on("error", reject); + if (init.body !== undefined) req.write(init.body); + req.end(); + }); +} + +async function howlFrameFetch(url, method, body, headers) { var u; - try { u = new URL(String(url)); } catch (e) { howlFrameFetchDeny(); } + try { u = howlFrameFetchURL(url); } catch (e) { howlFrameFetchDeny(); } if (!howlFrameFetchAllowed(u)) howlFrameFetchDeny(); - var init = { method: method, redirect: "manual" }; + var init = { method: method, redirect: "manual", headers: Object.assign({}, headers || {}) }; if (arguments.length >= 3) { init.body = body; } + var first = u; for (var hop = 0; ; hop++) { if (!(await howlFrameFetchResolvedOk(u))) howlFrameFetchDeny(); - var pending = fetch(u.href, init); + var pending = howlFrameFetchRequest(u, init); var r = await pending; if (r && r.type === "opaqueredirect") howlFrameFetchDeny(); var loc = r && r.status >= 300 && r.status < 400 && r.headers && r.headers.get ? r.headers.get("location") : null; if (!loc) return r.text(); if (hop >= 10) howlFrameFetchDeny(); - try { u = new URL(loc, u); } catch (e) { howlFrameFetchDeny(); } + var previous = u; + try { u = howlFrameFetchURL(loc, u); } catch (e) { howlFrameFetchDeny(); } + if (previous.protocol === "https:" && u.protocol === "http:") howlFrameFetchDeny(); + var optHost = u.hostname.replace(/^\[|\]$/g, ""); + if ((/^\[.*\]$/.test(u.hostname) || /^\d+\.\d+\.\d+\.\d+$/.test(optHost) || optHost === "localhost") && u.hostname !== first.hostname) howlFrameFetchDeny(); + if (previous.origin !== u.origin) { + Object.keys(init.headers || {}).forEach(function (name) { + if (/^(authorization|cookie)$/i.test(name)) delete init.headers[name]; + }); + } if (!howlFrameFetchAllowed(u)) howlFrameFetchDeny(); if (r.status === 303 || (r.status === 301 || r.status === 302) && init.method === "POST") { init = { method: "GET", redirect: "manual" }; @@ -1336,3 +1456,25 @@ func generateJSStatementRaw(node *ast.Node, reqVar string, depth int) string { // ast.ReportError(fmt.Sprintf("Unknown statement for JS: %s", head), node.Line, node.Column) return "" } + +// specialPrefixJSSource renders capability.SpecialPrefixes, the table shared +// by every target, as [16-byte prefix, bits] pairs (IPv4 rows as mapped IPv6). +func specialPrefixJSSource() string { + var b strings.Builder + b.WriteString("var howlFrameSpecialPrefixes = [\n") + for _, raw := range capability.SpecialPrefixes { + p := netip.MustParsePrefix(raw) + bits := p.Bits() + if p.Addr().Is4() { + bits += 96 + } + a := p.Addr().As16() + parts := make([]string, len(a)) + for i, x := range a { + parts[i] = strconv.Itoa(int(x)) + } + fmt.Fprintf(&b, " [[%s], %d], // %s\n", strings.Join(parts, ","), bits, raw) + } + b.WriteString("];\n\n") + return b.String() +} diff --git a/internal/backend/javascript/javascript_test.go b/internal/backend/javascript/javascript_test.go index f30bd648..5e68d240 100644 --- a/internal/backend/javascript/javascript_test.go +++ b/internal/backend/javascript/javascript_test.go @@ -103,7 +103,7 @@ func TestGenerateJSFetchWithBody(t *testing.T) { appCode, _ := GenerateJSCode(root) for _, want := range []string{ `howlFrameFetch("http://example.test/tasks", "POST", "{\"title\":\"Ship\"}")`, - `var pending = fetch(u.href, init);`, + `var pending = howlFrameFetchRequest(u, init);`, `howlFrameParseJSON(response)`, } { if !strings.Contains(appCode, want) { diff --git a/internal/backend/javascript/network_security_test.go b/internal/backend/javascript/network_security_test.go new file mode 100644 index 00000000..231e0514 --- /dev/null +++ b/internal/backend/javascript/network_security_test.go @@ -0,0 +1,253 @@ +package javascript + +import ( + "io" + "net" + "net/http" + "net/http/httptest" + "strconv" + "strings" + "sync/atomic" + "testing" + + "github.com/howlcipher/howlframe/internal/nettest" +) + +func securityJS(t *testing.T, script string, env ...string) { + t.Helper() + out, err, code := runJS(t, fetchJSHelper()+execJSHelper()+` +const assert = require("assert"); +(async () => { +`+script+` +})().then(()=>console.log("passed"),err=>{console.error(err);process.exit(1)}); +`, env...) + if code != 0 || strings.TrimSpace(out) != "passed" { + t.Fatalf("Node conformance: code=%d out=%q err=%s", code, out, err) + } +} + +const jsServers = ` +const http = require("http"); +const listen = async handler => { + const s = http.createServer(handler); + await new Promise(resolve=>s.listen(0,"127.0.0.1",resolve)); + return s; +}; +const url = s => "http://127.0.0.1:"+s.address().port; +const grants = (...urls) => {process.env.HOWLFRAME_ALLOW_CAPS=urls.map(u=>"network:fetch="+new URL(u).host).join(",");}; +const deny = async fn => {await assert.rejects(fn,/CAPABILITY_DENIED/);}; +` + +func TestJSRedirectCap11(t *testing.T) { + securityJS(t, jsServers+` + let hits=0; + const s=await listen((req,res)=>{ + hits++; const n=Number(req.url.slice(1)); + if(n===11) return res.end("escaped"); + res.writeHead(302,{location:"/"+(n+1)});res.end(); + }); + try { + grants(url(s));await deny(()=>howlFrameFetch(url(s)+"/0","GET")); + assert.equal(hits,11); + } finally {s.close();} +`) +} +func TestJSNamePrivateDial(t *testing.T) { + securityJS(t, jsServers+` + let hits=0; + const s=await listen((req,res)=>{hits++;res.end("ok");}); + const dns=require("dns"); + const original=dns.promises.lookup; + dns.promises.lookup=async()=>[{address:"127.0.0.1",family:4}]; + try { + const named="http://private.test:"+s.address().port; + grants(named);await deny(()=>howlFrameFetch(named,"GET"));assert.equal(hits,0); + grants("http://localhost:"+s.address().port); + assert.equal(await howlFrameFetch("http://localhost:"+s.address().port,"GET"),"ok"); + grants(url(s));assert.equal(await howlFrameFetch(url(s),"GET"),"ok"); + assert.equal(hits,2); + } finally {dns.promises.lookup=original;s.close();} +`) +} +func TestJSPrivateRanges(t *testing.T) { + list := func(xs []string) string { return `["` + strings.Join(xs, `","`) + `"]` } + securityJS(t, ` + for (const ip of `+list(append(nettest.BlockedAddresses, "0:0:0:0:0:0:0:1", "::ffff:7f00:1"))+`) { + assert.equal(howlFrameForbiddenAddress(ip,false),true,ip); + } + for (const ip of `+list(nettest.PublicAddresses)+`) assert.equal(howlFrameForbiddenAddress(ip,false),false,ip); +`) +} +func TestJSRedirectToOptOut(t *testing.T) { + securityJS(t, jsServers+` + const dns=require("dns"); const original=dns.promises.lookup; + dns.promises.lookup=async()=>[{address:"93.184.216.34",family:4}]; + let seen=[]; + howlFrameFetchRequest=async (u)=>{seen.push(u.href); const to=u.searchParams.get("to"); + return to ? {status:302,headers:{get:()=>to}} : {status:200,headers:{get:()=>null},text:async()=>"ok"};}; + process.env.HOWLFRAME_ALLOW_CAPS="network:fetch=api.example.com,network:fetch=10.0.0.5,network:fetch=localhost,network:fetch=127.0.0.1,network:fetch=[fd00::5]"; + try { + for (const [from,to] of [["http://api.example.com/","http://10.0.0.5/"],["http://api.example.com/","http://localhost/"],["http://api.example.com/","http://[fd00::5]/"],["http://localhost/","http://127.0.0.1/"],["http://127.0.0.1/","http://localhost/"],["http://10.0.0.5/","http://127.0.0.1/"]]) { + seen=[]; await deny(()=>howlFrameFetch(from+"?to="+encodeURIComponent(to),"GET")); assert.equal(seen.length,1,from+" -> "+to); + } + for (const [from,to] of [["http://api.example.com/","http://api.example.com/next"],["http://10.0.0.5/","http://10.0.0.5/next"],["http://localhost/","http://localhost/next"]]) { + assert.equal(await howlFrameFetch(from+"?to="+encodeURIComponent(to),"GET"),"ok",from+" -> "+to); + } + } finally {dns.promises.lookup=original;} +`) +} +func TestJSMulticastDial(t *testing.T) { + securityJS(t, ` + for(const ip of ["224.0.0.1","239.255.255.255","[ff00::1]","[ff02::1]"]) { + process.env.HOWLFRAME_ALLOW_CAPS="network:fetch="+ip; + await assert.rejects(()=>howlFrameFetch("http://"+ip,"GET"),/CAPABILITY_DENIED/); + } +`) +} +func TestJSCredentialsPortChange(t *testing.T) { + securityJS(t, jsServers+` + let credentials, hits=0; + const dst=await listen((req,res)=>{hits++;credentials=req.headers.authorization||req.headers.cookie||"";res.end("ok");}); + const src=await listen((req,res)=>{res.writeHead(307,{location:url(dst)});res.end();}); + try { + grants(url(src),url(dst)); + assert.equal(await howlFrameFetch(url(src),"GET",undefined,{Authorization:"Bearer secret",Cookie:"session=secret"}),"ok"); + assert.equal(hits,1);assert.equal(credentials,""); + } finally {src.close();dst.close();} +`) +} +func TestJSBrowserHTTPSDowngrade(t *testing.T) { + // Browser-style manual responses avoid trusting a fake DNS or TLS certificate. + securityJS(t, ` + const vm=require("vm"); + const source=howlFrameFetch.toString(); + let calls=0; + const browser={ + URL, howlFrameFetchURL, + howlFrameFetchAllowed:()=>true, + howlFrameFetchResolvedOk:async()=>true, + howlFrameFetchDeny, + howlFrameFetchRequest:async()=>{calls++;return {status:302,headers:{get:()=>"http://example.test/"},text:async()=>"escaped"};} + }; + vm.runInNewContext(source,browser); + await assert.rejects(()=>browser.howlFrameFetch("https://example.test/","GET"),/CAPABILITY_DENIED/); + assert.equal(calls,1); +`) +} +func TestJSAmbiguousNumericGrant(t *testing.T) { + securityJS(t, ` + for(const spec of ["127.1","2130706433","0177.0.0.1","0x7f.0.0.1"]) { + process.env.HOWLFRAME_ALLOW_CAPS="network:fetch="+spec; + assert.equal(howlFrameFetchAllowed(new URL("http://127.0.0.1/")),false,spec); + } + process.env.HOWLFRAME_ALLOW_CAPS="network:fetch=127.0.0.1"; + for(const host of ["127.1","2130706433","0177.0.0.1","0x7f.0.0.1"]) { + assert.throws(()=>howlFrameFetchURL("http://"+host),/CAPABILITY_DENIED/); + } + assert.equal(howlFrameFetchAllowed(new URL("http://127.0.0.1/")),true); + for(const raw of ["http:127.1","http:///127.1","http:\\127.1"]) assert.throws(()=>howlFrameFetchURL(raw),/CAPABILITY_DENIED/); +`) +} +func TestJSCodeLoadingEnv(t *testing.T) { + securityJS(t, ` + for(const name of ["PYTHONPATH","NODE_OPTIONS","BASH_ENV","ENV","PERL5LIB","PERL5OPT","RUBYOPT","RUBYLIB","PYTHONSTARTUP","NODE_PATH"]) { + for(const raw of [name,name.toLowerCase()]) { + process.env[raw]="injected";process.env.HOWLFRAME_ALLOW_CAPS="process:env="+raw; + assert.deepEqual(howlFrameExecEnv(),{},raw);delete process.env[raw]; + } + } +`) +} +func TestJSProxyEnvIgnored(t *testing.T) { + securityJS(t, jsServers+` + let connections=0; + const trap=await listen((req,res)=>res.end("proxy")); + trap.on("connection",()=>connections++); + const dst=await listen((req,res)=>res.end("direct")); + for(const key of ["HTTP_PROXY","HTTPS_PROXY","http_proxy","https_proxy","ALL_PROXY"])process.env[key]=url(trap); + process.env.NODE_USE_ENV_PROXY="1";process.env.NO_PROXY="";process.env.no_proxy=""; + globalThis.fetch=async()=>{throw new Error("environment-proxy fetch selected");}; + try { + grants(url(dst));assert.equal(await howlFrameFetch(url(dst),"GET"),"direct"); + assert.equal(connections,0); + } finally {trap.close();dst.close();} +`) +} + +// TestJSProxyEnvAtStartup also runs under Node 24 when node is selected on PATH. +func TestJSProxyEnvAtStartup(t *testing.T) { + var connections atomic.Int64 + trap := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { io.WriteString(w, "proxy") })) + trap.Config.ConnState = func(conn net.Conn, state http.ConnState) { + if state == http.StateNew { + connections.Add(1) + conn.Close() + } + } + trap.Start() + defer trap.Close() + target := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { io.WriteString(w, "direct") })) + defer target.Close() + tlsTarget := httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { io.WriteString(w, "tls-direct") })) + defer tlsTarget.Close() + script := ` + const targets=[` + strconv.Quote(target.URL) + `,` + strconv.Quote(tlsTarget.URL) + `]; + for(const u of targets) { + process.env.HOWLFRAME_ALLOW_CAPS="network:fetch="+new URL(u).host; + assert.equal(await howlFrameFetch(u,"GET"),u.startsWith("https")?"tls-direct":"direct"); + } +` + securityJS(t, script, "HTTP_PROXY="+trap.URL, "HTTPS_PROXY="+trap.URL, "http_proxy="+trap.URL, "https_proxy="+trap.URL, "NO_PROXY=", "no_proxy=", "NODE_USE_ENV_PROXY=1", "NODE_TLS_REJECT_UNAUTHORIZED=0") + if connections.Load() != 0 { + t.Fatalf("proxy got %d connections", connections.Load()) + } +} + +func TestJSPinsVettedLookup(t *testing.T) { + securityJS(t, jsServers+` + const s=await listen((req,res)=>res.end("pinned")); + const originalRequest=http.request; + const dns=require("dns");const originalLookup=dns.promises.lookup; + let resolutions=0,connectedAddress; + dns.promises.lookup=async()=>{resolutions++;return [{address:resolutions===1?"93.184.216.34":"127.0.0.1",family:4}];}; + http.request=(u,options,handler)=>{ + assert.equal(typeof options.lookup,"function","checked lookup must be used at connect time"); + options.lookup(u.hostname,{all:true},(err,addresses)=>{ + assert.ifError(err);connectedAddress=addresses[0].address; + }); + return originalRequest(url(s),{method:options.method,agent:false},handler); + }; + try { + grants("http://public.test"); + assert.equal(await howlFrameFetch("http://public.test","GET"),"pinned"); + assert.equal(resolutions,1);assert.equal(connectedAddress,"93.184.216.34"); + } finally {http.request=originalRequest;dns.promises.lookup=originalLookup;s.close();} +`) +} + +func TestJSHTTPSDowngrade(t *testing.T) { + var hits atomic.Int64 + dst := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { hits.Add(1); io.WriteString(w, "escaped") })) + defer dst.Close() + src := httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { http.Redirect(w, r, dst.URL, 302) })) + defer src.Close() + securityJS(t, ` + process.env.HOWLFRAME_ALLOW_CAPS=`+strconv.Quote("network:fetch="+strings.TrimPrefix(src.URL, "https://")+",network:fetch="+strings.TrimPrefix(dst.URL, "http://"))+`; + await assert.rejects(()=>howlFrameFetch(`+strconv.Quote(src.URL)+`,"GET"),/CAPABILITY_DENIED/); +`, "NODE_TLS_REJECT_UNAUTHORIZED=0") + if hits.Load() != 0 { + t.Fatalf("downgrade reached HTTP target %d times", hits.Load()) + } +} + +func TestJSFetchCompressedResponse(t *testing.T) { + securityJS(t, jsServers+` + const zlib=require("zlib"); + for(const encoding of ["gzip","deflate","br"]) { + const data=Buffer.from("\ufeffcompressed response"); + const body=encoding==="gzip"?zlib.gzipSync(data):encoding==="deflate"?zlib.deflateSync(data):zlib.brotliCompressSync(data); + const s=await listen((req,res)=>{res.writeHead(200,{"content-encoding":encoding});res.end(body);}); + try {grants(url(s));assert.equal(await howlFrameFetch(url(s),"GET"),"compressed response");} finally {s.close();} + } +`) +} diff --git a/internal/capability/destinations.go b/internal/capability/destinations.go new file mode 100644 index 00000000..1c2daaed --- /dev/null +++ b/internal/capability/destinations.go @@ -0,0 +1,79 @@ +package capability + +import ( + "net/netip" + "net/url" +) + +// SpecialPrefixes is the single table of destinations a hostname grant may +// not reach (#172, #177). Generated Go and JavaScript render this same table, +// so the three targets cannot drift. IPv4 entries also match IPv4-mapped +// IPv6; IPv4 embedded in IPv4-compatible (::a.b.c.d) and 6to4 (2002::/16) +// addresses is decoded and checked against the IPv4 rows. NAT64 and Teredo +// are refused outright because the embedded address is attacker-chosen. +var SpecialPrefixes = []string{ + "0.0.0.0/8", "10.0.0.0/8", "100.64.0.0/10", "127.0.0.0/8", "169.254.0.0/16", + "172.16.0.0/12", "192.0.0.0/24", "192.0.2.0/24", "192.88.99.0/24", "192.168.0.0/16", + "198.18.0.0/15", "198.51.100.0/24", "203.0.113.0/24", "224.0.0.0/4", "240.0.0.0/4", + "::/128", "::1/128", "64:ff9b::/96", "64:ff9b:1::/48", "100::/64", "2001::/32", + "2001:db8::/32", "fc00::/7", "fe80::/10", "fec0::/10", "ff00::/8", +} + +var specialPrefixes = func() []netip.Prefix { + out := make([]netip.Prefix, len(SpecialPrefixes)) + for i, s := range SpecialPrefixes { + out[i] = netip.MustParsePrefix(s) + } + return out +}() + +// ForbiddenDestination reports whether a hostname grant may not connect to a. +func ForbiddenDestination(a netip.Addr) bool { + a = a.Unmap() + if a.Is6() { + b := a.As16() + if b[0] == 0x20 && b[1] == 0x02 { // 6to4 + if ForbiddenDestination(netip.AddrFrom4([4]byte{b[2], b[3], b[4], b[5]})) { + return true + } + } + compat := true + for _, x := range b[:12] { + compat = compat && x == 0 + } + if compat && !(b[12] == 0 && b[13] == 0 && b[14] == 0) { // ::a.b.c.d + return ForbiddenDestination(netip.AddrFrom4([4]byte{b[12], b[13], b[14], b[15]})) + } + } + if !a.IsGlobalUnicast() { + return true + } + for _, p := range specialPrefixes { + if p.Contains(a) { + return true + } + } + return false +} + +// optOutHost reports an explicit opt-out destination: an IP literal or localhost. +func optOutHost(u *url.URL) (string, bool) { + t, err := urlTarget(u) + if err != nil { + return "", false + } + return t.host, t.ip || t.host == "localhost" +} + +// redirectToOptOut refuses a redirect that lands on an opt-out destination +// other than the host the request started on: a hostname grant must not be +// able to bounce into a granted private IP or localhost, and localhost must +// not bounce into a granted 127.0.0.1. +func redirectToOptOut(first, next *url.URL) bool { + host, opt := optOutHost(next) + if !opt { + return false + } + start, err := urlTarget(first) + return err != nil || start.host != host +} diff --git a/internal/capability/grants.go b/internal/capability/grants.go index a151315f..a5fdacc7 100644 --- a/internal/capability/grants.go +++ b/internal/capability/grants.go @@ -17,7 +17,7 @@ func ParseGrant(raw string) (Capability, error) { } if name, ok := strings.CutPrefix(raw, EnvGrantPrefix); ok { if DynamicLoaderEnv(name) { - return None, fmt.Errorf("process:env= refuses dynamic-loader variables (LD_*, DYLD_*): %q", raw) + return None, fmt.Errorf("process:env= refuses code-loading variables (including LD_*, DYLD_*): %q", raw) } if !validEnvName(name) { return None, fmt.Errorf("unknown capability: %q", raw) diff --git a/internal/capability/network.go b/internal/capability/network.go index 6f1f020b..02375aff 100644 --- a/internal/capability/network.go +++ b/internal/capability/network.go @@ -27,8 +27,9 @@ import ( // // Every redirect hop is re-checked against the same allow-list, an IP // literal matches only a grant that names that same IP, and a granted DNS -// name may not resolve to a link-local or unspecified address (cloud -// metadata services live there). Proxies from the environment are ignored. +// name may not resolve to private, loopback, NAT64, link-local or unspecified +// addresses, except for the explicit localhost opt-out. Multicast is always +// refused. Proxies from the environment are ignored. const FetchGrantPrefix = "network:fetch=" // FetchDenied is the typed detail of a refused outbound request. Required is @@ -114,6 +115,9 @@ func normalizeHost(h string) (string, bool, error) { return canonicalIP(addr), true, nil } h = strings.ToLower(h) + if strings.HasSuffix(h, ".") { + return "", false, fmt.Errorf("invalid host name: trailing dot") + } if looksNumeric(h) { return "", false, fmt.Errorf("ambiguous numeric host") } @@ -267,24 +271,23 @@ func (g Grants) AuthorizeFetch(u *url.URL) (string, error) { return "", &FetchDenied{Reason: "network: fetch host is not in the allow-list", Required: FetchGrantPrefix + t.String()} } -// forbiddenResolved reports addresses a granted DNS name may not reach: -// link-local (169.254.0.0/16, fe80::/10, which hosts cloud metadata) and -// unspecified addresses. Granting the IP literal itself is the only way in. +// forbiddenResolved reports addresses a hostname grant may not reach by +// default. Explicit IP-literal and localhost grants opt out, except multicast. func forbiddenResolved(a netip.Addr) bool { a = a.Unmap() - return a.IsLinkLocalUnicast() || a.IsLinkLocalMulticast() || a.IsUnspecified() || a.IsMulticast() + return ForbiddenDestination(a) } // checkDial vets the resolved address a connection is about to use. A DNS -// name (not an IP literal) may not land on a link-local, multicast or -// unspecified address. +// name may not land on a forbidden destination. Explicit literal and +// localhost grants opt out of private-address checks, but never multicast. func checkDial(host string, literal bool, address string) error { ap, err := netip.ParseAddrPort(address) if err != nil { return &FetchDenied{Reason: "network: fetch dial address is invalid", Required: FetchGrantPrefix + host} } - if !literal && forbiddenResolved(ap.Addr()) { - return &FetchDenied{Reason: "network: granted host resolved to a link-local or unspecified address", Required: FetchGrantPrefix + canonicalIP(ap.Addr())} + if ap.Addr().IsMulticast() || !literal && !strings.EqualFold(host, "localhost") && forbiddenResolved(ap.Addr()) { + return &FetchDenied{Reason: "network: granted host resolved to a forbidden destination", Required: FetchGrantPrefix + canonicalIP(ap.Addr())} } return nil } @@ -293,8 +296,8 @@ func checkDial(host string, literal bool, address string) error { const MaxFetchRedirects = 10 // FetchClient returns an HTTP client that re-authorizes every redirect hop -// against g, ignores proxy settings, and refuses to connect a granted DNS -// name to a link-local or unspecified address. onHop is called for every +// against g, ignores proxy settings, and vets the resolved destination at +// connection time. onHop is called for every // redirect hop with the matched grant or the denial. A denial surfaces from // Do as an error that wraps *FetchDenied. func FetchClient(g Grants, onHop func(u *url.URL, grant string, err error)) *http.Client { @@ -314,6 +317,31 @@ func FetchClient(g Grants, onHop func(u *url.URL, grant string, err error)) *htt d.Control = func(_, address string, _ syscall.RawConn) error { return checkDial(host, literal, address) } + if !literal { + _, port, _ := net.SplitHostPort(addr) + addresses, err := fetchLookupIP(ctx, host) + if err != nil { + return nil, err + } + for _, address := range addresses { + target := net.JoinHostPort(address.IP.String(), port) + if err := checkDial(host, literal, target); err != nil { + return nil, err + } + } + var last error + for _, address := range addresses { + conn, err := d.DialContext(ctx, network, net.JoinHostPort(address.IP.String(), port)) + if err == nil { + return conn, nil + } + last = err + } + if last != nil { + return nil, last + } + return nil, &FetchDenied{Reason: "network: host resolved to no addresses"} + } return d.DialContext(ctx, network, addr) }, ForceAttemptHTTP2: true, @@ -329,6 +357,26 @@ func FetchClient(g Grants, onHop func(u *url.URL, grant string, err error)) *htt if len(via) > MaxFetchRedirects { return fmt.Errorf("stopped after %d redirects", MaxFetchRedirects) } + previous := via[len(via)-1].URL + if previous.Scheme == "https" && req.URL.Scheme == "http" { + return &FetchDenied{Reason: "network: HTTPS redirect downgrade refused"} + } + if redirectToOptOut(via[0].URL, req.URL) { + required := FetchGrantPrefix + "" + if t, err := urlTarget(req.URL); err == nil { + required = FetchGrantPrefix + t.String() + } + err := &FetchDenied{Reason: "network: redirect to an IP-literal or localhost destination refused", Required: required} + return err + } + changedOrigin := !sameFetchOrigin(previous, req.URL) + for i := 1; i < len(via); i++ { + changedOrigin = changedOrigin || !sameFetchOrigin(via[i-1].URL, via[i].URL) + } + if changedOrigin { + req.Header.Del("Authorization") + req.Header.Del("Cookie") + } grant, err := g.AuthorizeFetch(req.URL) if onHop != nil { onHop(req.URL, grant, err) @@ -347,6 +395,8 @@ func AsFetchDenied(err error) (*FetchDenied, bool) { return nil, false } +var fetchLookupIP = net.DefaultResolver.LookupIPAddr + var fetchTestTransport http.RoundTripper // SetFetchTransportForTest replaces the network transport of every fetch @@ -376,3 +426,22 @@ func FetchGrantForURL(rawURL string) (string, bool) { // whose URL is not a compile-time constant. It is not a valid grant: the // operator must replace it with network:fetch=host[:port] entries. const DynamicFetchRequirement = FetchGrantPrefix + "" + +// sameFetchOrigin includes scheme and effective port, not just hostname. +func sameFetchOrigin(a, b *url.URL) bool { + ta, ea := urlTarget(a) + tb, eb := urlTarget(b) + return ea == nil && eb == nil && strings.EqualFold(a.Scheme, b.Scheme) && ta.host == tb.host && ta.port == tb.port +} + +// FetchPrivateOptOut describes explicit authority to reach private destinations. +func FetchPrivateOptOut(grant string) string { + if !strings.HasPrefix(grant, FetchGrantPrefix) { + return "" + } + t, err := parseFetchSpec(strings.TrimPrefix(grant, FetchGrantPrefix)) + if err == nil && (t.ip || t.host == "localhost") { + return "explicit IP-literal or localhost grant" + } + return "" +} diff --git a/internal/capability/network_security_test.go b/internal/capability/network_security_test.go new file mode 100644 index 00000000..c3d45eb9 --- /dev/null +++ b/internal/capability/network_security_test.go @@ -0,0 +1,55 @@ +package capability + +import ( + "net" + "net/http" + "net/netip" + "strings" + "testing" + + "github.com/howlcipher/howlframe/internal/nettest" +) + +func TestFetchSecurity(t *testing.T) { + nettest.Run(t, func(raw ...string) *http.Client { + var g Grants + for _, s := range raw { + g = append(g, Capability(s)) + } + return FetchClient(g, nil) + }, func(f nettest.Lookup) func() { + old := fetchLookupIP + fetchLookupIP = f + return func() { fetchLookupIP = old } + }) +} +func TestFetchPrivateRanges(t *testing.T) { + for _, ip := range append(nettest.BlockedAddresses, "224.0.0.1", "ff00::1") { + if err := checkDial("private.test", false, net.JoinHostPort(ip, "80")); err == nil { + t.Errorf("allowed %s", ip) + } + } + for _, ip := range nettest.PublicAddresses { + if forbiddenResolved(netip.MustParseAddr(ip)) { + t.Errorf("blocked public %s", ip) + } + } +} +func TestFetchTrailingDotDiagnostic(t *testing.T) { + _, err := ParseGrant("network:fetch=example.com.") + if err == nil || !strings.Contains(err.Error(), "trailing dot") || strings.Contains(err.Error(), "numeric") { + t.Fatalf("inaccurate diagnostic: %v", err) + } +} +func TestEnvGrantRefusesCodeLoading(t *testing.T) { + for _, name := range []string{"PYTHONPATH", "NODE_OPTIONS", "BASH_ENV", "ENV", "PERL5LIB", "PERL5OPT", "RUBYOPT", "RUBYLIB", "PYTHONSTARTUP", "NODE_PATH"} { + for _, raw := range []string{name, strings.ToLower(name)} { + if _, err := ParseGrant(EnvGrantPrefix + raw); err == nil { + t.Errorf("allowed %s", raw) + } + if len((Grants{Capability(EnvGrantPrefix + raw)}).ExecEnv(func(string) (string, bool) { return "injected", true })) != 0 { + t.Errorf("raw grant allowed %s", raw) + } + } + } +} diff --git a/internal/capability/network_test.go b/internal/capability/network_test.go index 8262f2ad..94a359ea 100644 --- a/internal/capability/network_test.go +++ b/internal/capability/network_test.go @@ -147,7 +147,7 @@ func TestForbiddenResolved(t *testing.T) { t.Errorf("%s not forbidden", a) } } - for _, a := range []string{"127.0.0.1", "10.0.0.1", "93.184.216.34", "::1"} { + for _, a := range []string{"93.184.216.34", "2001:4860:4860::8888"} { if forbiddenResolved(netip.MustParseAddr(a)) { t.Errorf("%s forbidden", a) } diff --git a/internal/capability/process.go b/internal/capability/process.go index f4734b66..1b645e0c 100644 --- a/internal/capability/process.go +++ b/internal/capability/process.go @@ -92,11 +92,16 @@ func slotMatches(p, arg string) bool { const EnvGrantPrefix = "process:env=" // DynamicLoaderEnv reports variables that steer the dynamic loader -// (LD_PRELOAD, LD_LIBRARY_PATH, any LD_* and DYLD_*). Passing one to an exec +// or interpreter code loading (LD_*, DYLD_*, PYTHONPATH, NODE_OPTIONS, etc.). +// Passing one to an exec // child would let it load arbitrary code into an allow-listed program, so // process:env= refuses them. func DynamicLoaderEnv(name string) bool { u := strings.ToUpper(name) + switch u { + case "PYTHONPATH", "NODE_OPTIONS", "BASH_ENV", "ENV", "PERL5LIB", "PERL5OPT", "RUBYOPT", "RUBYLIB", "PYTHONSTARTUP", "NODE_PATH": + return true + } return strings.HasPrefix(u, "LD_") || strings.HasPrefix(u, "DYLD_") } diff --git a/internal/nettest/security.go b/internal/nettest/security.go new file mode 100644 index 00000000..6603a8a5 --- /dev/null +++ b/internal/nettest/security.go @@ -0,0 +1,268 @@ +// Package nettest contains consumer-driven network conformance assertions shared +// by the capability client and the emitted Go helper. +package nettest + +import ( + "context" + "crypto/tls" + "fmt" + "io" + "net" + "net/http" + "net/http/httptest" + "net/url" + "strings" + "sync/atomic" + "testing" + "time" +) + +type Lookup func(context.Context, string) ([]net.IPAddr, error) + +func Run(t *testing.T, client func(...string) *http.Client, lookup func(Lookup) func()) { + grant := func(raw string) string { u, _ := url.Parse(raw); return "network:fetch=" + u.Host } + t.Run("RedirectToOptOut", func(t *testing.T) { + c := client("network:fetch=api.example.com", "network:fetch=10.0.0.5", "network:fetch=localhost", "network:fetch=127.0.0.1", "network:fetch=[fd00::5]") + hop := func(from, to string) error { + first, _ := http.NewRequest("GET", from, nil) + next, _ := http.NewRequest("GET", to, nil) + return c.CheckRedirect(next, []*http.Request{first}) + } + for _, tc := range [][2]string{ + {"http://api.example.com/", "http://10.0.0.5/"}, + {"http://api.example.com/", "http://localhost/"}, + {"http://api.example.com/", "http://[fd00::5]/"}, + {"http://localhost/", "http://127.0.0.1/"}, + {"http://127.0.0.1/", "http://localhost/"}, + {"http://10.0.0.5/", "http://127.0.0.1/"}, + } { + if err := hop(tc[0], tc[1]); err == nil { + t.Errorf("redirect %s -> %s allowed", tc[0], tc[1]) + } + } + for _, tc := range [][2]string{ + {"http://api.example.com/", "http://api.example.com/next"}, + {"http://10.0.0.5/", "http://10.0.0.5/next"}, + {"http://localhost/", "http://localhost/next"}, + } { + if err := hop(tc[0], tc[1]); err != nil { + t.Errorf("redirect %s -> %s refused: %v", tc[0], tc[1], err) + } + } + }) + t.Run("RedirectCap11", func(t *testing.T) { + var hits atomic.Int64 + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + hits.Add(1) + var n int + fmt.Sscanf(r.URL.Path, "/%d", &n) + if n == 11 { + io.WriteString(w, "escaped") + return + } + http.Redirect(w, r, fmt.Sprintf("/%d", n+1), 302) + })) + defer srv.Close() + c := client(grant(srv.URL)) + resp, err := c.Get(srv.URL + "/0") + if resp != nil { + resp.Body.Close() + } + if err == nil || hits.Load() != 11 { + t.Fatalf("11-hop chain escaped cap: err=%v hits=%d", err, hits.Load()) + } + }) + t.Run("ProxyEnvIgnored", func(t *testing.T) { + var traps atomic.Int64 + trap := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { traps.Add(1) })) + trap.Config.ConnState = func(_ net.Conn, s http.ConnState) { + if s == http.StateNew { + traps.Add(1) + } + } + defer trap.Close() + for _, key := range []string{"HTTPS_PROXY", "HTTP_PROXY", "https_proxy", "http_proxy", "ALL_PROXY"} { + t.Setenv(key, trap.URL) + } + t.Setenv("NO_PROXY", "") + t.Setenv("no_proxy", "") + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { io.WriteString(w, "direct") })) + defer srv.Close() + restore := lookup(func(context.Context, string) ([]net.IPAddr, error) { + return []net.IPAddr{{IP: net.ParseIP("127.0.0.1")}}, nil + }) + defer restore() + // localhost is the named opt-out; net/http ProxyFromEnvironment bypasses it. + // An explicit non-loopback literal defeats that bypass without real DNS. + c := client("network:fetch=192.0.2.1:80") + tr := c.Transport.(*http.Transport) + original := tr.DialContext + tr.DialContext = func(ctx context.Context, network, addr string) (net.Conn, error) { + if strings.HasPrefix(addr, "192.0.2.1:") { + addr = strings.TrimPrefix(srv.URL, "http://") + } + return original(ctx, network, addr) + } + resp, err := c.Get("http://192.0.2.1/") + if err != nil { + t.Fatal(err) + } + b, _ := io.ReadAll(resp.Body) + resp.Body.Close() + if string(b) != "direct" || traps.Load() != 0 { + t.Fatalf("proxy used: body=%q trap connections=%d", b, traps.Load()) + } + // Also exercise HTTPS_PROXY with a TLS destination. + tlsSrv := httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { io.WriteString(w, "tls-direct") })) + defer tlsSrv.Close() + tr.TLSClientConfig = &tls.Config{InsecureSkipVerify: true} // local test certificate only + tr.DialContext = func(ctx context.Context, network, addr string) (net.Conn, error) { + if strings.HasPrefix(addr, "192.0.2.1:") { + addr = strings.TrimPrefix(tlsSrv.URL, "https://") + } + return original(ctx, network, addr) + } + resp, err = c.Get("https://192.0.2.1/") + if err != nil { + t.Fatal(err) + } + resp.Body.Close() + if traps.Load() != 0 { + t.Fatalf("HTTPS proxy received %d connections", traps.Load()) + } + }) + t.Run("HTTPSDowngrade", func(t *testing.T) { + var hits atomic.Int64 + dst := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { hits.Add(1) })) + defer dst.Close() + src := httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { http.Redirect(w, r, dst.URL, 302) })) + defer src.Close() + c := client(grant(src.URL), grant(dst.URL)) + c.Transport.(*http.Transport).TLSClientConfig = &tls.Config{InsecureSkipVerify: true} + resp, err := c.Get(src.URL) + if resp != nil { + resp.Body.Close() + } + if err == nil || hits.Load() != 0 { + t.Fatalf("downgrade connected: err=%v hits=%d", err, hits.Load()) + } + }) + t.Run("CredentialsPortChange", func(t *testing.T) { + var credentials string + var hits atomic.Int64 + dst := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + credentials += r.Header.Get("Authorization") + r.Header.Get("Cookie") + hits.Add(1) + if r.URL.Path != "/final" { + http.Redirect(w, r, "/final", 307) + } + })) + defer dst.Close() + src := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { http.Redirect(w, r, dst.URL, 307) })) + defer src.Close() + c := client(grant(src.URL), grant(dst.URL)) + req, _ := http.NewRequest("GET", src.URL, nil) + req.Header.Set("Authorization", "Bearer secret") + req.Header.Set("Cookie", "session=secret") + resp, err := c.Do(req) + if err != nil { + t.Fatal(err) + } + resp.Body.Close() + if hits.Load() != 2 || credentials != "" { + t.Fatalf("credentials crossed port: hits=%d credentials=%q", hits.Load(), credentials) + } + }) + + t.Run("CredentialsSameOrigin", func(t *testing.T) { + c := client("network:fetch=example.test") + first, _ := http.NewRequest("GET", "http://EXAMPLE.test/", nil) + next, _ := http.NewRequest("GET", "http://example.test:80/next", nil) + next.Header.Set("Authorization", "Bearer secret") + next.Header.Set("Cookie", "session=secret") + if err := c.CheckRedirect(next, []*http.Request{first}); err != nil { + t.Fatal(err) + } + if next.Header.Get("Authorization") != "Bearer secret" || next.Header.Get("Cookie") != "session=secret" { + t.Fatal("same effective origin lost credentials") + } + }) + t.Run("NamePrivateDial", func(t *testing.T) { + var hits atomic.Int64 + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { hits.Add(1) })) + defer srv.Close() + restore := lookup(func(context.Context, string) ([]net.IPAddr, error) { + return []net.IPAddr{{IP: net.ParseIP("127.0.0.1")}}, nil + }) + defer restore() + port := strings.TrimPrefix(srv.URL, "http://127.0.0.1:") + c := client("network:fetch=private.test:" + port) + resp, err := c.Get("http://private.test:" + port) + if resp != nil { + resp.Body.Close() + } + if err == nil || hits.Load() != 0 { + t.Fatalf("name connected to loopback: err=%v hits=%d", err, hits.Load()) + } + c = client("network:fetch=localhost:" + port) + resp, err = c.Get("http://localhost:" + port) + if err != nil { + t.Fatalf("localhost opt-out: %v", err) + } + resp.Body.Close() + c = client(grant(srv.URL)) + resp, err = c.Get(srv.URL) + if err != nil { + t.Fatalf("literal opt-out: %v", err) + } + resp.Body.Close() + if hits.Load() != 2 { + t.Fatalf("opt-outs failed: %d", hits.Load()) + } + }) + + t.Run("PrivateRangesDial", func(t *testing.T) { + for _, ip := range BlockedAddresses { + t.Run(ip, func(t *testing.T) { + restore := lookup(func(context.Context, string) ([]net.IPAddr, error) { return []net.IPAddr{{IP: net.ParseIP(ip)}}, nil }) + defer restore() + c := client("network:fetch=private.test:80") + ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond) + defer cancel() + conn, err := c.Transport.(*http.Transport).DialContext(ctx, "tcp", "private.test:80") + if conn != nil { + conn.Close() + } + if err == nil || !(strings.Contains(err.Error(), "network:") || strings.Contains(err.Error(), "capability denied")) { + t.Fatalf("missing private policy denial for %s: %v", ip, err) + } + }) + } + }) + t.Run("MulticastDial", func(t *testing.T) { + for _, address := range []string{"224.0.0.1:80", "[ff02::1]:80"} { + c := client("network:fetch=" + address) + _, err := c.Transport.(*http.Transport).DialContext(context.Background(), "tcp", address) + if err == nil || !(strings.Contains(err.Error(), "network:") || strings.Contains(err.Error(), "capability denied")) { + t.Fatalf("multicast lacks policy denial: %v", err) + } + } + }) +} + +// BlockedAddresses must be refused for a hostname grant on every target. +var BlockedAddresses = []string{ + "127.1.2.3", "::1", "10.1.2.3", "172.16.0.1", "172.31.255.255", "192.168.1.1", + "fc00::1", "fd12::1", "64:ff9b::7f00:1", "64:ff9b::808:808", "64:ff9b:1::1", "64:ff9b:1::808:808", + "169.254.169.254", "fe80::1", "::ffff:127.0.0.1", "224.0.0.1", "ff00::1", + "100.64.0.1", "100.100.100.200", "100.127.255.255", "::ffff:100.100.100.200", + "0.1.2.3", "198.18.0.1", "198.19.255.255", "192.0.0.1", "240.0.0.1", "255.255.255.255", + "fec0::1", "::7f00:1", "::a00:5", "2002:7f00:1::1", "2002:a00:5::1", "2002:6464:64c8::1", + "2001:0:4136:e378::1", "100::1", "2001:db8::1", "::", +} + +// PublicAddresses must stay reachable for a hostname grant on every target. +var PublicAddresses = []string{ + "8.8.8.8", "172.15.0.1", "172.32.0.1", "192.169.1.1", "100.63.255.255", "100.128.0.1", + "198.17.255.255", "198.20.0.1", "2001:4860::1", "2606:4700::1", "2002:808:808::1", "::808:808", +} diff --git a/internal/vm/receipt.go b/internal/vm/receipt.go index 6a1f91c6..52139e64 100644 --- a/internal/vm/receipt.go +++ b/internal/vm/receipt.go @@ -44,7 +44,8 @@ type ReceiptEffect struct { // for a denied fetch, the network:fetch=host:port grant it would need. // Allow-list entries are copied verbatim from the normalized grant, so // pattern literals (exec argument literals, hosts) appear as written. - Grant string `json:"grant,omitempty"` + Grant string `json:"grant,omitempty"` + PrivateDestinationOptOut string `json:"private_destination_opt_out,omitempty"` } type ReceiptExit struct { Status string `json:"status"` @@ -114,7 +115,7 @@ func (vm *BCVM) recordEffectGrant(cap capability.Capability, op bytecode.Opcode, r.truncated = true return } - r.effects = append(r.effects, ReceiptEffect{bytecode.Registry[op].Name, string(cap), d.summary, decision, grant}) + r.effects = append(r.effects, ReceiptEffect{bytecode.Registry[op].Name, string(cap), d.summary, decision, grant, receiptPrivateOptOut(decision, grant)}) } // recordHop records a redirect hop. Unlike recordEffectGrant it does not @@ -134,7 +135,7 @@ func (vm *BCVM) recordHop(cap capability.Capability, op bytecode.Opcode, decisio r.truncated = true return } - r.effects = append(r.effects, ReceiptEffect{bytecode.Registry[op].Name, string(cap), "redirect", decision, grant}) + r.effects = append(r.effects, ReceiptEffect{bytecode.Registry[op].Name, string(cap), "redirect", decision, grant, receiptPrivateOptOut(decision, grant)}) } // RunBytecodeWithReceipt returns normally on every VM outcome. Diagnostics match @@ -243,3 +244,10 @@ func (vm *BCVM) effectSummary(inst bytecode.BCInstruction, env *BcEnv) string { } return summary } + +func receiptPrivateOptOut(decision, grant string) string { + if decision != "allowed" { + return "" + } + return capability.FetchPrivateOptOut(grant) +} diff --git a/internal/vm/receipt_test.go b/internal/vm/receipt_test.go index 1e77eb5b..06d126a1 100644 --- a/internal/vm/receipt_test.go +++ b/internal/vm/receipt_test.go @@ -57,7 +57,7 @@ func testReceiptFetch(t *testing.T, origin string) { address := strings.Replace(origin, "http://", "http://user:pw@", 1) + "/path?token=secret#frag" grant := fetchGrantFor(t, origin) r, raw, _ := receiptRun(t, fmt.Sprintf(`(cli_app (fetch %q "GET"))`, address), DefaultExecutionPolicy(), grant) - if r.Exit.Code != 0 || len(r.Effects) != 1 || r.Effects[0] != (ReceiptEffect{"FETCH", "network", origin, "allowed", string(grant)}) { + if r.Exit.Code != 0 || len(r.Effects) != 1 || r.Effects[0] != (ReceiptEffect{"FETCH", "network", origin, "allowed", string(grant), capability.FetchPrivateOptOut(string(grant))}) { t.Fatalf("receipt: %s", raw) } for _, secret := range []string{"secret", "token", "pw", "/path", "user:"} { @@ -269,3 +269,23 @@ func TestReceiptDeadlineUnits(t *testing.T) { } } } + +func TestReceiptFetchPrivateOptOut(t *testing.T) { + defer capability.SetFetchTransportForTest(receiptRoundTripFunc(func(r *http.Request) (*http.Response, error) { + return &http.Response{StatusCode: 200, Body: io.NopCloser(strings.NewReader("ok")), Header: make(http.Header)}, nil + }))() + for _, host := range []string{"127.0.0.1", "localhost", "api.example.com"} { + grant := capability.Capability("network:fetch=" + host + ":8099") + r, raw, _ := receiptRun(t, fmt.Sprintf(`(cli_app (fetch "http://%s:8099/" "GET"))`, host), DefaultExecutionPolicy(), grant) + if r.Exit.Code != 0 || len(r.Effects) != 1 { + t.Fatalf("receipt: %s", raw) + } + want := "" + if host != "api.example.com" { + want = "explicit IP-literal or localhost grant" + } + if r.Effects[0].PrivateDestinationOptOut != want { + t.Fatalf("%s opt-out = %q, want %q", host, r.Effects[0].PrivateDestinationOptOut, want) + } + } +}