HFIR adapter resource bounds: shared lowering, depth/size caps, run deadline (#169, #170)

change_log.md

@@ -3,6 +3,7 @@
 ## Unreleased
 
 ### Breaking
+* HFIR model adapter resource bounds (#169, #170): shared closed scalar `const`/`binary` subexpressions now lower once into private functions instead of exponentially expanded fragments; effects, variable reads and allocating expressions retain their evaluation sites and order. Candidates exceeding 2048 dependency levels, 65536 expanded nodes or 4194304 aggregate cached-fragment nodes fail closed with `HFIR_TRANSPORT_TOO_DEEP` / `HFIR_TRANSPORT_TOO_LARGE`, including callers that construct `Candidate` directly. Breaking for adapter consumers: in-memory programs returned by `CompileCandidate` now have a maximum one-second wall-clock budget (a shorter runner deadline still applies); a sleep beyond the remaining budget is refused immediately with `HFIR_DEADLINE_EXCEEDED`, which `try_let` cannot catch. Receipt paths report both the effective budget and typed failure. The adapter constraint is runner-owned and ephemeral; HFBC serialization remains unchanged, and artifact runners still select their execution policy. Explicit-deadline standalone VM sleeps also refuse durations beyond their remaining budget with the existing `LIMIT_EXCEEDED` code. Production HFIR compilation, #90 (Partial), target fences, SHA-256 pins, freeze tests and the Assurance tip-lock remain unchanged. Journal: [HFIR adapter resource blockers](docs/journals/2026-10-13_hfir_adapter_resource_blockers.md).
 * HFIR model adapter (#88) now covers the full `-compile-hfir-bc` language. `hfir-model-adapter/v1` accepts every node kind `LowerToBytecode` lowers from source (`hfir.AdapterNodeKinds`): `defun`/`param`/`call`/`return`, `while`, `for`, `try`/`catch`, `read_file`, `write_file`, `mkdir`, `exec`, `fetch`, `parse_json`, `is_nil`, `map_keys`, `html_escape`, `attr_escape`, `regex_match`, `time_now`, `sleep` (transport role `duration`), `cli_args`, `req_query`/`req_header`/`req_path`, and `(map_get <expr> key)` via a `target` edge, on top of the Phase-1 kinds. `for`/`while` control edges are derived from validated roles like `if`; the schema still has no control-edge field. Kinds the source lowerer does not lower (`spawn`, `spawn_agent`, `grants`, `llm_generate`, `http_server`/`res*`, store/db forms, `read_line`, `http_request`, `match`, ...) still fail with `HFIR_TRANSPORT_KIND`. No new opcode, capability, grant, lowering or language form; effects reached through the adapter are checked by the same run-time grants and fail closed without them. Breaking for adapter clients: candidate bounds rise to 4096 nodes, 64 inputs per node and 1 MiB per transport (from 128 / 32 / 64 KiB); the transport type check now types BOOL consts as bool, `to_int`/`to_float` as numbers and `str_split` as a list, and accepts string `+` string, so a candidate mixing a bool into arithmetic, or a `sleep` whose duration is a known non-number, is now rejected with `HFIR_TRANSPORT_TYPE`; a `return` outside a `defun` body is rejected with `HFIR_TRANSPORT_RETURN` unless it is a straight-line main-level statement (program/sequence/let body or if branch), which source already allows; cycles are rejected (`HFIR_TRANSPORT_CYCLE`) before type inference; non-const nodes never carry `literal_kind`. New experimental flag `-hfir-adapter-roundtrip` (with `-compile-hfir-bc`) sends the source graph through the adapter (encode, strict decode, `CompileCandidate`). `TestModelAdapterCoversArityTable` requires every one of the 386 corpus programs `-compile-hfir-bc` lowers to round-trip byte for byte, and every arity-table form to be covered (126 of 159) or carry an exemption with a rejected sample (33) that the test checks stays outside `-compile-hfir-bc`. The prod-flip criteria's model-adapter row is updated; the promote-blocker fence, production `-compile-bc`, #90 (Partial) and the tip-lock are unchanged. Pinned token benchmarks unchanged.
 * Network scoping (#165, #166). HowlFrame is a capability-governed runtime, not a sandbox; see `docs/reference/network_authority.md`. (1) The coarse `network` grant no longer authorizes outbound `fetch` / `http_request` on any path (interpreter, `-run-bc` for AST and HFIR artifacts, generated Go and JavaScript). Each host needs `network:fetch=<host>[:<port>]`; matching is exact (no wildcard or suffix match, IP literals only match the same canonical IP, ambiguous numeric hosts such as `127.1` are refused, an omitted port means the scheme default). Every redirect hop is re-checked, a granted DNS name may not connect to a link-local/unspecified address (cloud metadata), and proxy environment variables are ignored. Migration: replace `-allow-caps network` with `-allow-caps network,network:fetch=api.example.com` (keep `network` only if the program also serves HTTP or calls the model forms). (2) Route handlers no longer keep `network` implicitly: a route holds only its `(grants ...)` list or the server default. `res`, `res_json`, `res_header`, `http_res_header` and `req_method` no longer require any capability (`RES`, `RES_JSON`, `HTTP_RES_HEADER` and `HTTP_REQ_METHOD` now have an empty capability field). A route that fetches must list `network:fetch=` hosts, which the runner must hold (`apps/webhook_relay` now grants its upstream explicitly). (3) A denied fetch raises `CAPABILITY_DENIED` and the receipt's denied `FETCH`/`HTTP_REQUEST` effect carries `grant: "network:fetch=host:port"`; allowed fetches record the matched grant, and each redirect hop is its own effect. (4) Generated Go refuses `fetch`/`http_request` inside a route or middleware with `UNSUPPORTED_ON_TARGET`, because it cannot scope a single route. (5) A failing route handler now returns a generic `500 {"error":"internal server error"}` to the client; the runtime-error JSON is written to the server log (and receipt) only. (6) `process:env=` refuses dynamic-loader variables (`LD_*`, including `LD_PRELOAD`, and `DYLD_*`, case-insensitive) on every path, and generated Go/JS never pass them to exec children. (7) `-required-caps` reports `network:fetch=host:port` for constant-URL fetches and `network:fetch=<dynamic>` for computed URLs instead of `network`. Receipt `grant` strings are copied verbatim from the normalized grant, including `process:exec=` pattern literals; do not put secrets in grant patterns.
 * Authority hardening (process allow-lists and S9 attenuation). HowlFrame is a capability-governed runtime, not a sandbox. (1) A bare `process` grant no longer authorizes `exec` on any path (interpreter, `-run-bc` for AST and HFIR artifacts, generated Go/JS): each program needs `process:exec=<absolute path>[ <pattern>...]` with exact-argv matching (`literal`, `*`, trailing `**`). Bare names (no PATH lookup), relative or non-clean paths, and symlinked program paths are refused. Migrate `-allow-caps process` + `(exec "find" ...)` to `-allow-caps 'process:exec=/usr/bin/find **'` + `(exec "/usr/bin/find" ...)`. (2) `spawn_agent` children now start with **no grants** instead of inheriting the parent's; `(spawn_agent name (grants "cap" ...) task)` hands an explicit subset, which must be covered by the parent, so nested spawns cannot escalate. A nested spawner needs `(grants "process")`. Spawning still requires coarse `process`. (3) Denials are typed `CAPABILITY_DENIED` with a reason and are recorded in the `--receipt` (EXEC / SPAWN_AGENT, decision `denied`). New `grants` construct (opaque, arity 0+). No new opcode; the child grant list rides in `SPAWN_AGENT` `string_operand_2`. Pinned benchmark results unchanged; #90 stays Partial; no production flip. Reference: `docs/reference/process_authority.md`. Review follow-ups: (4) a `*` slot no longer matches an empty argument or one starting with `-` (grant flags as literals; `\*` matches a literal `*`; `**` still accepts anything). (5) exec children start with an empty environment on every path; `process:env=NAME` passes one variable through. (6) `spawn` bodies and HTTP route handlers are attenuated like `spawn_agent`: `(spawn (grants ...) (lambda ...))` (default none) and `(route path (grants ...) (lambda ...))` (default `network` only, which a handler needs to respond; one `(grants ...)` directly under `http_server` sets the default for its routes); over-requests fail closed at `SPAWN`/`HTTP_ROUTE`. Apps updated: counter_api (server-wide `(grants "database")`, plus two equivalent simplifications to stay under the Python token count: `n` is initialized at startup, so `nz` fallbacks were dropped, and `/inc` binds the string once), rest_tasks, task_api (`database`), status_api, webhook_relay (`environment`); task_api's network-only run now fails at startup instead of per request. Receipts carry the matched exec grant; spawn denials no longer echo the requested grant; generated Go/JS refuse a non-final `**`. Interpreter denials are typed but still write no receipt (documented).

docs/journals/2026-10-13_hfir_adapter_resource_blockers.md

@@ -0,0 +1,26 @@
+# HFIR model adapter resource blockers (#169, #170)
+
+Consumer: public `DecodeCandidate` / `CompileCandidate` boundary and the VM evidence/receipt runners under `DefaultExecutionPolicy`.
+
+## Findings and changes
+
+The fragment cache memoized lowering but still copied expanded instruction slices at each consumer. Shared closed scalar expressions now have collision-free private functions, lowering each shared node once. Calls remain at the original evaluation sites: this shares compilation, not execution results. Variable reads, allocation, control flow and effects are not outlined. A shared effect-containing sequence still prints `1,2,1,2`, in order.
+
+Memoized saturating expansion accounting rejects dependency depth above 2048, entry expansion above 65536 nodes, or aggregate cached-fragment expansion above 4194304 nodes. Both JSON intake and direct `Candidate` compilation enforce the gate. The depth limit preserves the existing 1500-binding let-chain fixture, required by `TestModelAdapterCoversArityTable`; the 4096-node proposal with a 4093-operation chain is rejected promptly. These limits bound lowering without widening language or authority.
+
+Programs returned directly by `CompileCandidate` carry an ephemeral runner-owned adapter constraint. VM evidence and receipt runners apply a maximum one-second wall-clock deadline, respecting shorter caller deadlines. Sleep compares milliseconds with the remaining budget before converting to `time.Duration`, so `1e15` milliseconds cannot overflow the conversion or wait unboundedly. Adapter budget failures are `HFIR_DEADLINE_EXCEEDED` and escape `try_let`. Receipts retain the effective deadline and typed failure. HFBC serialization is unchanged: serialized artifacts retain the existing artifact-runner policy contract, rather than transporting runner authority.
+
+## Executable evidence
+
+* `TestAdapterSharedDepth22Bounded`: successful depth-22 JSON proposal compilation, fewer than 150 instructions, under one second and 32 MiB allocated. Observed local run: 1.89 ms and 578488 allocated bytes. A smaller shared graph executes to the expected arithmetic result.
+* `TestAdapterDepthCapRejects4095Chain`: over-depth proposals, including the maximum-size chain, reject with `HFIR_TRANSPORT_TOO_DEEP` in under one second.
+* `TestAdapterExpansionCap`: shared allocating expressions reject with `HFIR_TRANSPORT_TOO_LARGE`. `TestAdapterAggregateFragmentCap` separately rejects excessive summed fragment storage even when entry expansion and depth are within bounds.
+* `TestAdapterDirectCandidateBounds`: direct graph construction cannot bypass intake bounds.
+* `TestAdapterRunDeadlineAndSleep/{huge_sleep,busy_loop,caught_sleep,normal_sleep}`: enormous sleep returns promptly; a busy loop exceeds a five-millisecond budget and fails closed; catch cannot turn refusal into success; ordinary sleep succeeds. Every receipt reports the effective deadline.
+* `TestAdapterSharedEffectsKeepOrder`: repeated effects preserve multiplicity and order.
+
+Ten temporary mutations were tested individually and restored: disabling shared lowering, depth rejection, entry-size rejection, aggregate-fragment rejection, the direct compilation gate, default adapter deadline, sleep-budget refusal, deadline propagation through catch, receipt effective-budget reporting, and loop deadline checking. Every corresponding negative test failed; the default-deadline and sharing removals hit the three-second test timeout, bounding the mutation experiment itself.
+
+Compatibility run: `TestHFIRBenchmarkSolutionParity` passed all 268 programs (0/268 mismatches); `TestModelAdapterCoversArityTable` passed all 386 corpus round-trips, with 126/159 forms covered and 33 fenced; `TestProdFlipCriteriaLock` and CLI resource-flag tests passed. `go build ./...` and `go vet ./...` passed. Final `go test ./...` passed, with no ignored failures.
+
+No production HFIR flip, no change to #90 (Partial), no tip-lock refresh, and no DOM additions. Pins, freeze tests, Go/JS target fences and governed checks remain intact.

internal/bytecode/bytecode.go

@@ -52,9 +52,10 @@ func (inst *BCInstruction) UnmarshalJSON(data []byte) error {
 }
 
 type BCProgram struct {
-	Version   int                    `json:"version"`
-	Functions map[string]*BCFunction `json:"functions"`
-	Main      []BCInstruction        `json:"main"`
+	modelAdapter bool
+	Version      int                    `json:"version"`
+	Functions    map[string]*BCFunction `json:"functions"`
+	Main         []BCInstruction        `json:"main"`
 
 	// mainOrigins is an ephemeral direct-lowering sidecar. It is deliberately
 	// unexported: provenance is useful only while the trusted lowerer program
@@ -997,3 +998,7 @@ func compileSpawnGrants(node *ast.Node) string {
 	}
 	return strings.Join(out, "\n")
 }
+
+// MarkModelAdapter attaches a trusted, in-memory runner constraint.
+func (p *BCProgram) MarkModelAdapter()    { p.modelAdapter = true }
+func (p *BCProgram) IsModelAdapter() bool { return p != nil && p.modelAdapter }