diff --git a/change_log.md b/change_log.md index e28f3601..c4cc7894 100644 --- a/change_log.md +++ b/change_log.md @@ -3,6 +3,7 @@ ## Unreleased ### Breaking +* HFIR model adapter resource bounds (#169, #170): shared closed scalar `const`/`binary` subexpressions now lower once into private functions instead of exponentially expanded fragments; effects, variable reads and allocating expressions retain their evaluation sites and order. Candidates exceeding 2048 dependency levels, 65536 expanded nodes or 4194304 aggregate cached-fragment nodes fail closed with `HFIR_TRANSPORT_TOO_DEEP` / `HFIR_TRANSPORT_TOO_LARGE`, including callers that construct `Candidate` directly. Breaking for adapter consumers: in-memory programs returned by `CompileCandidate` now have a maximum one-second wall-clock budget (a shorter runner deadline still applies); a sleep beyond the remaining budget is refused immediately with `HFIR_DEADLINE_EXCEEDED`, which `try_let` cannot catch. Receipt paths report both the effective budget and typed failure. The adapter constraint is runner-owned and ephemeral; HFBC serialization remains unchanged, and artifact runners still select their execution policy. Explicit-deadline standalone VM sleeps also refuse durations beyond their remaining budget with the existing `LIMIT_EXCEEDED` code. Production HFIR compilation, #90 (Partial), target fences, SHA-256 pins, freeze tests and the Assurance tip-lock remain unchanged. Journal: [HFIR adapter resource blockers](docs/journals/2026-10-13_hfir_adapter_resource_blockers.md). * HFIR model adapter (#88) now covers the full `-compile-hfir-bc` language. `hfir-model-adapter/v1` accepts every node kind `LowerToBytecode` lowers from source (`hfir.AdapterNodeKinds`): `defun`/`param`/`call`/`return`, `while`, `for`, `try`/`catch`, `read_file`, `write_file`, `mkdir`, `exec`, `fetch`, `parse_json`, `is_nil`, `map_keys`, `html_escape`, `attr_escape`, `regex_match`, `time_now`, `sleep` (transport role `duration`), `cli_args`, `req_query`/`req_header`/`req_path`, and `(map_get key)` via a `target` edge, on top of the Phase-1 kinds. `for`/`while` control edges are derived from validated roles like `if`; the schema still has no control-edge field. Kinds the source lowerer does not lower (`spawn`, `spawn_agent`, `grants`, `llm_generate`, `http_server`/`res*`, store/db forms, `read_line`, `http_request`, `match`, ...) still fail with `HFIR_TRANSPORT_KIND`. No new opcode, capability, grant, lowering or language form; effects reached through the adapter are checked by the same run-time grants and fail closed without them. Breaking for adapter clients: candidate bounds rise to 4096 nodes, 64 inputs per node and 1 MiB per transport (from 128 / 32 / 64 KiB); the transport type check now types BOOL consts as bool, `to_int`/`to_float` as numbers and `str_split` as a list, and accepts string `+` string, so a candidate mixing a bool into arithmetic, or a `sleep` whose duration is a known non-number, is now rejected with `HFIR_TRANSPORT_TYPE`; a `return` outside a `defun` body is rejected with `HFIR_TRANSPORT_RETURN` unless it is a straight-line main-level statement (program/sequence/let body or if branch), which source already allows; cycles are rejected (`HFIR_TRANSPORT_CYCLE`) before type inference; non-const nodes never carry `literal_kind`. New experimental flag `-hfir-adapter-roundtrip` (with `-compile-hfir-bc`) sends the source graph through the adapter (encode, strict decode, `CompileCandidate`). `TestModelAdapterCoversArityTable` requires every one of the 386 corpus programs `-compile-hfir-bc` lowers to round-trip byte for byte, and every arity-table form to be covered (126 of 159) or carry an exemption with a rejected sample (33) that the test checks stays outside `-compile-hfir-bc`. The prod-flip criteria's model-adapter row is updated; the promote-blocker fence, production `-compile-bc`, #90 (Partial) and the tip-lock are unchanged. Pinned token benchmarks unchanged. * Network scoping (#165, #166). HowlFrame is a capability-governed runtime, not a sandbox; see `docs/reference/network_authority.md`. (1) The coarse `network` grant no longer authorizes outbound `fetch` / `http_request` on any path (interpreter, `-run-bc` for AST and HFIR artifacts, generated Go and JavaScript). Each host needs `network:fetch=[:]`; matching is exact (no wildcard or suffix match, IP literals only match the same canonical IP, ambiguous numeric hosts such as `127.1` are refused, an omitted port means the scheme default). Every redirect hop is re-checked, a granted DNS name may not connect to a link-local/unspecified address (cloud metadata), and proxy environment variables are ignored. Migration: replace `-allow-caps network` with `-allow-caps network,network:fetch=api.example.com` (keep `network` only if the program also serves HTTP or calls the model forms). (2) Route handlers no longer keep `network` implicitly: a route holds only its `(grants ...)` list or the server default. `res`, `res_json`, `res_header`, `http_res_header` and `req_method` no longer require any capability (`RES`, `RES_JSON`, `HTTP_RES_HEADER` and `HTTP_REQ_METHOD` now have an empty capability field). A route that fetches must list `network:fetch=` hosts, which the runner must hold (`apps/webhook_relay` now grants its upstream explicitly). (3) A denied fetch raises `CAPABILITY_DENIED` and the receipt's denied `FETCH`/`HTTP_REQUEST` effect carries `grant: "network:fetch=host:port"`; allowed fetches record the matched grant, and each redirect hop is its own effect. (4) Generated Go refuses `fetch`/`http_request` inside a route or middleware with `UNSUPPORTED_ON_TARGET`, because it cannot scope a single route. (5) A failing route handler now returns a generic `500 {"error":"internal server error"}` to the client; the runtime-error JSON is written to the server log (and receipt) only. (6) `process:env=` refuses dynamic-loader variables (`LD_*`, including `LD_PRELOAD`, and `DYLD_*`, case-insensitive) on every path, and generated Go/JS never pass them to exec children. (7) `-required-caps` reports `network:fetch=host:port` for constant-URL fetches and `network:fetch=` for computed URLs instead of `network`. Receipt `grant` strings are copied verbatim from the normalized grant, including `process:exec=` pattern literals; do not put secrets in grant patterns. * Authority hardening (process allow-lists and S9 attenuation). HowlFrame is a capability-governed runtime, not a sandbox. (1) A bare `process` grant no longer authorizes `exec` on any path (interpreter, `-run-bc` for AST and HFIR artifacts, generated Go/JS): each program needs `process:exec=[ ...]` with exact-argv matching (`literal`, `*`, trailing `**`). Bare names (no PATH lookup), relative or non-clean paths, and symlinked program paths are refused. Migrate `-allow-caps process` + `(exec "find" ...)` to `-allow-caps 'process:exec=/usr/bin/find **'` + `(exec "/usr/bin/find" ...)`. (2) `spawn_agent` children now start with **no grants** instead of inheriting the parent's; `(spawn_agent name (grants "cap" ...) task)` hands an explicit subset, which must be covered by the parent, so nested spawns cannot escalate. A nested spawner needs `(grants "process")`. Spawning still requires coarse `process`. (3) Denials are typed `CAPABILITY_DENIED` with a reason and are recorded in the `--receipt` (EXEC / SPAWN_AGENT, decision `denied`). New `grants` construct (opaque, arity 0+). No new opcode; the child grant list rides in `SPAWN_AGENT` `string_operand_2`. Pinned benchmark results unchanged; #90 stays Partial; no production flip. Reference: `docs/reference/process_authority.md`. Review follow-ups: (4) a `*` slot no longer matches an empty argument or one starting with `-` (grant flags as literals; `\*` matches a literal `*`; `**` still accepts anything). (5) exec children start with an empty environment on every path; `process:env=NAME` passes one variable through. (6) `spawn` bodies and HTTP route handlers are attenuated like `spawn_agent`: `(spawn (grants ...) (lambda ...))` (default none) and `(route path (grants ...) (lambda ...))` (default `network` only, which a handler needs to respond; one `(grants ...)` directly under `http_server` sets the default for its routes); over-requests fail closed at `SPAWN`/`HTTP_ROUTE`. Apps updated: counter_api (server-wide `(grants "database")`, plus two equivalent simplifications to stay under the Python token count: `n` is initialized at startup, so `nz` fallbacks were dropped, and `/inc` binds the string once), rest_tasks, task_api (`database`), status_api, webhook_relay (`environment`); task_api's network-only run now fails at startup instead of per request. Receipts carry the matched exec grant; spawn denials no longer echo the requested grant; generated Go/JS refuse a non-final `**`. Interpreter denials are typed but still write no receipt (documented). diff --git a/docs/journals/2026-10-13_hfir_adapter_resource_blockers.md b/docs/journals/2026-10-13_hfir_adapter_resource_blockers.md new file mode 100644 index 00000000..c3a78d64 --- /dev/null +++ b/docs/journals/2026-10-13_hfir_adapter_resource_blockers.md @@ -0,0 +1,26 @@ +# HFIR model adapter resource blockers (#169, #170) + +Consumer: public `DecodeCandidate` / `CompileCandidate` boundary and the VM evidence/receipt runners under `DefaultExecutionPolicy`. + +## Findings and changes + +The fragment cache memoized lowering but still copied expanded instruction slices at each consumer. Shared closed scalar expressions now have collision-free private functions, lowering each shared node once. Calls remain at the original evaluation sites: this shares compilation, not execution results. Variable reads, allocation, control flow and effects are not outlined. A shared effect-containing sequence still prints `1,2,1,2`, in order. + +Memoized saturating expansion accounting rejects dependency depth above 2048, entry expansion above 65536 nodes, or aggregate cached-fragment expansion above 4194304 nodes. Both JSON intake and direct `Candidate` compilation enforce the gate. The depth limit preserves the existing 1500-binding let-chain fixture, required by `TestModelAdapterCoversArityTable`; the 4096-node proposal with a 4093-operation chain is rejected promptly. These limits bound lowering without widening language or authority. + +Programs returned directly by `CompileCandidate` carry an ephemeral runner-owned adapter constraint. VM evidence and receipt runners apply a maximum one-second wall-clock deadline, respecting shorter caller deadlines. Sleep compares milliseconds with the remaining budget before converting to `time.Duration`, so `1e15` milliseconds cannot overflow the conversion or wait unboundedly. Adapter budget failures are `HFIR_DEADLINE_EXCEEDED` and escape `try_let`. Receipts retain the effective deadline and typed failure. HFBC serialization is unchanged: serialized artifacts retain the existing artifact-runner policy contract, rather than transporting runner authority. + +## Executable evidence + +* `TestAdapterSharedDepth22Bounded`: successful depth-22 JSON proposal compilation, fewer than 150 instructions, under one second and 32 MiB allocated. Observed local run: 1.89 ms and 578488 allocated bytes. A smaller shared graph executes to the expected arithmetic result. +* `TestAdapterDepthCapRejects4095Chain`: over-depth proposals, including the maximum-size chain, reject with `HFIR_TRANSPORT_TOO_DEEP` in under one second. +* `TestAdapterExpansionCap`: shared allocating expressions reject with `HFIR_TRANSPORT_TOO_LARGE`. `TestAdapterAggregateFragmentCap` separately rejects excessive summed fragment storage even when entry expansion and depth are within bounds. +* `TestAdapterDirectCandidateBounds`: direct graph construction cannot bypass intake bounds. +* `TestAdapterRunDeadlineAndSleep/{huge_sleep,busy_loop,caught_sleep,normal_sleep}`: enormous sleep returns promptly; a busy loop exceeds a five-millisecond budget and fails closed; catch cannot turn refusal into success; ordinary sleep succeeds. Every receipt reports the effective deadline. +* `TestAdapterSharedEffectsKeepOrder`: repeated effects preserve multiplicity and order. + +Ten temporary mutations were tested individually and restored: disabling shared lowering, depth rejection, entry-size rejection, aggregate-fragment rejection, the direct compilation gate, default adapter deadline, sleep-budget refusal, deadline propagation through catch, receipt effective-budget reporting, and loop deadline checking. Every corresponding negative test failed; the default-deadline and sharing removals hit the three-second test timeout, bounding the mutation experiment itself. + +Compatibility run: `TestHFIRBenchmarkSolutionParity` passed all 268 programs (0/268 mismatches); `TestModelAdapterCoversArityTable` passed all 386 corpus round-trips, with 126/159 forms covered and 33 fenced; `TestProdFlipCriteriaLock` and CLI resource-flag tests passed. `go build ./...` and `go vet ./...` passed. Final `go test ./...` passed, with no ignored failures. + +No production HFIR flip, no change to #90 (Partial), no tip-lock refresh, and no DOM additions. Pins, freeze tests, Go/JS target fences and governed checks remain intact. diff --git a/internal/bytecode/bytecode.go b/internal/bytecode/bytecode.go index b0e71472..a75368a6 100644 --- a/internal/bytecode/bytecode.go +++ b/internal/bytecode/bytecode.go @@ -52,9 +52,10 @@ func (inst *BCInstruction) UnmarshalJSON(data []byte) error { } type BCProgram struct { - Version int `json:"version"` - Functions map[string]*BCFunction `json:"functions"` - Main []BCInstruction `json:"main"` + modelAdapter bool + Version int `json:"version"` + Functions map[string]*BCFunction `json:"functions"` + Main []BCInstruction `json:"main"` // mainOrigins is an ephemeral direct-lowering sidecar. It is deliberately // unexported: provenance is useful only while the trusted lowerer program @@ -997,3 +998,7 @@ func compileSpawnGrants(node *ast.Node) string { } return strings.Join(out, "\n") } + +// MarkModelAdapter attaches a trusted, in-memory runner constraint. +func (p *BCProgram) MarkModelAdapter() { p.modelAdapter = true } +func (p *BCProgram) IsModelAdapter() bool { return p != nil && p.modelAdapter } diff --git a/internal/hfir/adapter_blockers_test.go b/internal/hfir/adapter_blockers_test.go new file mode 100644 index 00000000..2121a6fb --- /dev/null +++ b/internal/hfir/adapter_blockers_test.go @@ -0,0 +1,209 @@ +package hfir + +import ( + "bytes" + "encoding/json" + "fmt" + "runtime" + "strings" + "testing" + "time" + + "github.com/howlcipher/howlframe/internal/vm" +) + +func blockerDAG(depth int, shared bool) candidateTransport { + t := candidateTransport{SchemaVersion: ModelAdapterSchemaVersion, GraphVersion: "v1", EntryNode: "program"} + t.Nodes = append(t.Nodes, testNode("n0", "const", "1", "INT", nil)) + for i := 1; i <= depth; i++ { + edges := []transportEdge{{Role: "left", NodeID: NodeID(fmt.Sprintf("n%d", i-1))}, {Role: "right", NodeID: "n0"}} + if shared { + edges[1].NodeID = edges[0].NodeID + } + t.Nodes = append(t.Nodes, testNode(NodeID(fmt.Sprintf("n%d", i)), "binary", "+", "", edges)) + } + t.Nodes = append(t.Nodes, testNode("print", "print", "", "", []transportEdge{{Role: "value", NodeID: NodeID(fmt.Sprintf("n%d", depth))}}), testNode("program", "program", "", "", []transportEdge{{Role: "body", NodeID: "print"}})) + return t +} + +func TestAdapterSharedDepth22Bounded(t *testing.T) { + data, _ := json.Marshal(blockerDAG(22, true)) + runtime.GC() + var before, after runtime.MemStats + runtime.ReadMemStats(&before) + start := time.Now() + candidate, ds := DecodeCandidate(data) + if len(ds) > 0 { + t.Fatal(ds) + } + p, ds := CompileCandidate(candidate) + if len(ds) > 0 { + t.Fatal(ds) + } + runtime.ReadMemStats(&after) + t.Logf("depth-22 lowering: %s, %d allocated bytes", time.Since(start), after.TotalAlloc-before.TotalAlloc) + if time.Since(start) > time.Second || after.TotalAlloc-before.TotalAlloc > 32<<20 { + t.Fatalf("unbounded lowering: %s, %d bytes", time.Since(start), after.TotalAlloc-before.TotalAlloc) + } + count := len(p.Main) + for _, f := range p.Functions { + count += len(f.Instructions) + } + if count > 150 { + t.Fatalf("expanded shared DAG: %d instructions", count) + } + // A small shared DAG proves the outlined calls preserve the arithmetic result. + c := mustCandidate(t, blockerDAG(4, true)) + p, ds = CompileCandidate(c) + if len(ds) > 0 { + t.Fatal(ds) + } + var out, errout bytes.Buffer + ev := vm.RunBytecodeWithEvidence(p, nil, vm.DefaultExecutionPolicy(), nil, strings.NewReader(""), &out, &errout, 0) + if ev.RuntimeFailure != nil || out.String() != "16\n" { + t.Fatalf("semantics: %#v %q", ev, out.String()) + } +} + +func TestAdapterDepthCapRejects4095Chain(t *testing.T) { + for _, depth := range []int{2048, 4092, 4093} { + data, _ := json.Marshal(blockerDAG(depth, false)) + start := time.Now() + c, ds := DecodeCandidate(data) + if c.Graph != nil || len(ds) != 1 || ds[0].Code != "HFIR_TRANSPORT_TOO_DEEP" { + t.Fatalf("depth %d: %#v", depth, ds) + } + if time.Since(start) > time.Second { + t.Fatal("depth rejection too slow") + } + } +} + +func TestAdapterExpansionCap(t *testing.T) { + // Shared allocation expressions are deliberately not outlined. + tr := blockerDAG(17, true) + for i := 1; i < len(tr.Nodes)-2; i++ { + tr.Nodes[i].Kind = "list" + tr.Nodes[i].Value = "" + for j := range tr.Nodes[i].Inputs { + tr.Nodes[i].Inputs[j].Role = "item" + } + } + data, _ := json.Marshal(tr) + c, ds := DecodeCandidate(data) + if c.Graph != nil || len(ds) != 1 || ds[0].Code != "HFIR_TRANSPORT_TOO_LARGE" { + t.Fatalf("expansion: %#v", ds) + } +} + +func TestAdapterRunDeadlineAndSleep(t *testing.T) { + for _, tc := range []struct { + name, source string + deadline time.Duration + fail bool + }{ + {"huge_sleep", `(cli_app (sleep 1000000000000000))`, 0, true}, + {"busy_loop", `(cli_app (while true (do)))`, 5 * time.Millisecond, true}, + {"caught_sleep", `(cli_app (try_let (x (sleep 1000000000000000)) (catch e (print "caught")) (print x)))`, 0, true}, + {"normal_sleep", `(cli_app (sleep 2) (print 7))`, 0, false}, + } { + t.Run(tc.name, func(t *testing.T) { + g := parseAndLower(t, tc.source, "deadline.howl", false) + data, err := EncodeCandidateTransport(g) + if err != nil { + t.Fatal(err) + } + c, ds := DecodeCandidate(data) + if len(ds) > 0 { + t.Fatal(ds) + } + p, ds := CompileCandidate(c) + if len(ds) > 0 { + t.Fatal(ds) + } + policy := vm.DefaultExecutionPolicy() + policy.Deadline = tc.deadline + if tc.name == "busy_loop" { + policy.Limits.MaxInstructions = 1000000000 + } + var out, stderr bytes.Buffer + start := time.Now() + ev, receipt := vm.RunBytecodeWithReceipt(p, nil, policy, nil, strings.NewReader(""), &out, &stderr, "test", "test") + if tc.deadline == 0 && receipt.Limits.DeadlineMS != 1000 { + t.Fatalf("receipt omits effective deadline: %#v", receipt.Limits) + } + _ = ev + raw, _ := json.Marshal(receipt) + if tc.fail { + if receipt.Exit.Error == nil || receipt.Exit.Error.Code != "HFIR_DEADLINE_EXCEEDED" { + t.Fatalf("receipt: %s", raw) + } + } else if out.String() != "7\n" { + t.Fatalf("normal sleep: %q, receipt %s", out.String(), raw) + } + if time.Since(start) > time.Second { + t.Fatal("deadline did not return promptly") + } + }) + } +} + +func TestAdapterSharedEffectsKeepOrder(t *testing.T) { + tr := candidateTransport{SchemaVersion: ModelAdapterSchemaVersion, GraphVersion: "v1", EntryNode: "program", Nodes: []transportNode{ + testNode("program", "program", "", "", []transportEdge{{Role: "body", NodeID: "seq"}, {Role: "body", NodeID: "seq"}}), + testNode("seq", "sequence", "", "", []transportEdge{{Role: "body", NodeID: "p1"}, {Role: "body", NodeID: "p2"}}), + testNode("p1", "print", "", "", []transportEdge{{Role: "value", NodeID: "one"}}), + testNode("p2", "print", "", "", []transportEdge{{Role: "value", NodeID: "two"}}), + testNode("one", "const", "1", "INT", nil), testNode("two", "const", "2", "INT", nil), + }} + c := mustCandidate(t, tr) + p, ds := CompileCandidate(c) + if len(ds) > 0 { + t.Fatal(ds) + } + var out, stderr bytes.Buffer + ev := vm.RunBytecodeWithEvidence(p, nil, vm.DefaultExecutionPolicy(), nil, nil, &out, &stderr, 0) + if ev.RuntimeFailure != nil || out.String() != "1\n2\n1\n2\n" { + t.Fatalf("effect order: %#v %q", ev, out.String()) + } +} + +func TestAdapterDirectCandidateBounds(t *testing.T) { + tr := blockerDAG(2048, false) + g := NewGraph() + g.EntryNode = tr.EntryNode + for _, n := range tr.Nodes { + g.AddNode(nodeFromTransport(n)) + } + p, ds := CompileCandidate(Candidate{Graph: g}) + if p != nil || len(ds) != 1 || ds[0].Code != "HFIR_TRANSPORT_TOO_DEEP" { + t.Fatalf("direct candidate bypassed transport bounds: %#v", ds) + } +} + +func TestAdapterAggregateFragmentCap(t *testing.T) { + tr := blockerDAG(1700, false) + second := blockerDAG(1700, false) + for _, n := range second.Nodes { + if n.ID == "n0" || n.ID == "program" { + continue + } + n.ID += "b" + for i := range n.Inputs { + if n.Inputs[i].NodeID != "n0" { + n.Inputs[i].NodeID += "b" + } + } + tr.Nodes = append(tr.Nodes, n) + } + for i := range tr.Nodes { + if tr.Nodes[i].ID == "program" { + tr.Nodes[i].Inputs = append(tr.Nodes[i].Inputs, transportEdge{Role: "body", NodeID: "printb"}) + } + } + data, _ := json.Marshal(tr) + c, ds := DecodeCandidate(data) + if c.Graph != nil || len(ds) != 1 || ds[0].Code != "HFIR_TRANSPORT_TOO_LARGE" { + t.Fatalf("aggregate fragments: %#v", ds) + } +} diff --git a/internal/hfir/adapter_limits.go b/internal/hfir/adapter_limits.go new file mode 100644 index 00000000..6c45d649 --- /dev/null +++ b/internal/hfir/adapter_limits.go @@ -0,0 +1,109 @@ +package hfir + +import "fmt" + +const maxAdapterDepth = 2048 +const maxAdapterExpansion = 65536 + +// Only closed scalar expressions are outlined. Variable reads, allocations, +// control flow and effects retain their original evaluation sites and order. +func adapterSharedPure(graph *Graph) map[NodeID]string { + t := transportFromGraph(graph) + byID := map[NodeID]transportNode{} + uses := map[NodeID]int{} + names := map[string]bool{} + for _, n := range t.Nodes { + byID[n.ID] = n + names[n.Value] = true + for _, e := range n.Inputs { + uses[e.NodeID]++ + } + } + known := map[NodeID]bool{} + visited := map[NodeID]bool{} + var pure func(NodeID) bool + pure = func(id NodeID) bool { + if visited[id] { + return known[id] + } + visited[id] = true + n := byID[id] + ok := n.Kind == "const" || n.Kind == "binary" + for _, e := range n.Inputs { + if !pure(e.NodeID) { + ok = false + } + } + known[id] = ok + return ok + } + result := map[NodeID]string{} + for _, n := range t.Nodes { + if uses[n.ID] > 1 && n.Kind != "const" && pure(n.ID) { + name := fmt.Sprintf("__hfir_shared_%s", n.ID) + for names[name] { + name += "_" + } + names[name] = true + result[n.ID] = name + } + } + return result +} + +// Count expansion with saturating arithmetic before recursive validation and +// fragment allocation. Depth is independent of the amount of DAG sharing. +func validateAdapterExpansion(t candidateTransport) *Diagnostic { + g := NewGraph() + for _, n := range t.Nodes { + g.AddNode(nodeFromTransport(n)) + } + g.EntryNode = t.EntryNode + shared := adapterSharedPure(g) + byID := map[NodeID]transportNode{} + for _, n := range t.Nodes { + byID[n.ID] = n + } + type cost struct{ depth, size int } + memo := map[NodeID]cost{} + var walk func(NodeID) cost + walk = func(id NodeID) cost { + if c, ok := memo[id]; ok { + return c + } + c := cost{1, 1} + for _, e := range byID[id].Inputs { + child := walk(e.NodeID) + if child.depth+1 > c.depth { + c.depth = child.depth + 1 + } + size := child.size + if _, ok := shared[e.NodeID]; ok { + size = 1 + } + c.size += size + if c.size > maxAdapterExpansion { + c.size = maxAdapterExpansion + 1 + } + } + memo[id] = c + return c + } + c := walk(t.EntryNode) + if c.depth > maxAdapterDepth { + d := adapterDiagnostic("HFIR_TRANSPORT_TOO_DEEP", "candidate exceeds 2048 dependency levels", t.EntryNode) + return &d + } + total := 0 + for _, fragment := range memo { + total += fragment.size + if total > 4*1024*1024 { + break + } + } + if c.size > maxAdapterExpansion || total > 4*1024*1024 { + d := adapterDiagnostic("HFIR_TRANSPORT_TOO_LARGE", "candidate fragment expansion exceeds 65536 nodes", t.EntryNode) + return &d + } + return nil +} diff --git a/internal/hfir/bytecode.go b/internal/hfir/bytecode.go index 9757f985..8ff55829 100644 --- a/internal/hfir/bytecode.go +++ b/internal/hfir/bytecode.go @@ -23,8 +23,13 @@ func LowerToBytecode(graph *Graph) (*bytecode.BCProgram, []Diagnostic) { // LowerToBytecodeIncremental converts the Phase-1 semantic HFIR subset into a // BCProgram, reusing cached bytecode fragments for preserved subgraphs. func LowerToBytecodeIncremental(graph *Graph, preservedCache map[NodeID][]bytecode.BCInstruction) (*bytecode.BCProgram, map[NodeID][]bytecode.BCInstruction, int, []Diagnostic) { + return lowerToBytecode(graph, preservedCache, nil) +} + +func lowerToBytecode(graph *Graph, preservedCache map[NodeID][]bytecode.BCInstruction, shared map[NodeID]string) (*bytecode.BCProgram, map[NodeID][]bytecode.BCInstruction, int, []Diagnostic) { compiler := &bytecodeLowerer{ graph: graph, + shared: shared, compiling: make(map[NodeID]bool), cachedNodes: preservedCache, reusedNodes: make(map[NodeID]bool), @@ -53,13 +58,15 @@ func LowerToBytecodeIncremental(graph *Graph, preservedCache map[NodeID][]byteco } type bytecodeLowerer struct { - graph *Graph - prog *bytecode.BCProgram - compiling map[NodeID]bool - cachedNodes map[NodeID][]bytecode.BCInstruction - reusedNodes map[NodeID]bool - nodeInsts map[NodeID][]bytecode.BCInstruction - loweredCount int + shared map[NodeID]string + buildingShared map[NodeID]bool + graph *Graph + prog *bytecode.BCProgram + compiling map[NodeID]bool + cachedNodes map[NodeID][]bytecode.BCInstruction + reusedNodes map[NodeID]bool + nodeInsts map[NodeID][]bytecode.BCInstruction + loweredCount int } func (c *bytecodeLowerer) compile(node *Node) (instructions []bytecode.BCInstruction, diagnostic *Diagnostic) { @@ -67,6 +74,24 @@ func (c *bytecodeLowerer) compile(node *Node) (instructions []bytecode.BCInstruc value := c.diagnostic(nil, "HFIR node is missing") return nil, &value } + if name, ok := c.shared[node.ID]; ok && !c.buildingShared[node.ID] { + if c.prog.Functions[name] == nil { + if c.buildingShared == nil { + c.buildingShared = map[NodeID]bool{} + } + c.buildingShared[node.ID] = true + body, d := c.compile(node) + delete(c.buildingShared, node.ID) + if d != nil { + return nil, d + } + body = append(body, instruction(bytecode.OpReturn, "RETURN", nil)) + c.prog.Functions[name] = &bytecode.BCFunction{Name: name, Instructions: body} + } + inst := instruction(bytecode.OpCall, "CALL", func(i *bytecode.BCInstruction) { i.StringOperand = name }) + bytecode.SetSemanticOrigin(&inst, string(node.ID)) + return []bytecode.BCInstruction{inst}, nil + } // A defun registers a BCFunction. That side effect is not stored in the // instruction cache, so a preserved defun is lowered again and the // function table is rebuilt. while has no function-table side effect. diff --git a/internal/hfir/model_adapter.go b/internal/hfir/model_adapter.go index e23bf806..40fb44f8 100644 --- a/internal/hfir/model_adapter.go +++ b/internal/hfir/model_adapter.go @@ -183,16 +183,20 @@ func CompileCandidate(candidate Candidate) (*bytecode.BCProgram, []Diagnostic) { if candidate.Graph == nil { return nil, []Diagnostic{adapterDiagnostic("HFIR_TRANSPORT_INVALID", "candidate graph is required", "")} } + if diagnostic := validateTransport(transportFromGraph(candidate.Graph)); diagnostic != nil { + return nil, []Diagnostic{*diagnostic} + } if diagnostics := NewVerifier(candidate.Graph, TargetBytecode).Verify(); hasErrors(diagnostics) { return nil, diagnostics } - program, diagnostics := LowerToBytecode(candidate.Graph) + program, _, _, diagnostics := lowerToBytecode(candidate.Graph, nil, adapterSharedPure(candidate.Graph)) if len(diagnostics) != 0 { return nil, diagnostics } if err := bytecode.ValidateProgram(program); err != nil { return nil, []Diagnostic{adapterDiagnostic("HFIR_BYTECODE_INVALID", err.Error(), candidate.Graph.EntryNode)} } + program.MarkModelAdapter() return program, nil } @@ -619,6 +623,9 @@ func validateTransport(transport candidateTransport) *Diagnostic { diagnostic := adapterDiagnostic("HFIR_TRANSPORT_CYCLE", "cyclic data dependency is not executable", cycle) return &diagnostic } + if diagnostic := validateAdapterExpansion(transport); diagnostic != nil { + return diagnostic + } if diagnostic := validateReturnPlacement(transport); diagnostic != nil { return diagnostic } diff --git a/internal/vm/receipt.go b/internal/vm/receipt.go index 79e8fc28..6a1f91c6 100644 --- a/internal/vm/receipt.go +++ b/internal/vm/receipt.go @@ -140,6 +140,7 @@ func (vm *BCVM) recordHop(cap capability.Capability, op bytecode.Opcode, decisio // RunBytecodeWithReceipt returns normally on every VM outcome. Diagnostics match // RunBytecodeWithPolicy; the caller must persist the receipt before process exit. func RunBytecodeWithReceipt(prog *bytecode.BCProgram, args []string, policy ExecutionPolicy, caps []capability.Capability, in io.Reader, out, errOut io.Writer, artifactSHA256, compilerVersion string) (int, ExecutionReceipt) { + policy = effectiveExecutionPolicy(prog, policy) recorder := &receiptRecorder{effects: []ReceiptEffect{}} ev := runBytecodeEvidence(prog, args, policy, caps, in, out, errOut, 0, recorder) grant := []string{} diff --git a/internal/vm/vm.go b/internal/vm/vm.go index a849b2de..1222cd58 100644 --- a/internal/vm/vm.go +++ b/internal/vm/vm.go @@ -1841,6 +1841,7 @@ func RunBytecodeWithEvidence(prog *bytecode.BCProgram, cliArgs []string, policy } func runBytecodeEvidence(prog *bytecode.BCProgram, cliArgs []string, policy ExecutionPolicy, allowedCaps []capability.Capability, in io.Reader, out io.Writer, errOut io.Writer, traceLimit int, recorder *receiptRecorder) (evidence bytecode.ExecutionEvidence) { + policy = effectiveExecutionPolicy(prog, policy) ctx := context.Background() if policy.Deadline > 0 { var cancel context.CancelFunc @@ -2243,6 +2244,9 @@ func (vm *BCVM) run(insts []bytecode.BCInstruction, env *BcEnv) any { if _, ok := r.(VmReturn); ok { panic(r) } + if e, ok := r.(*VMError); ok && e.Code == "HFIR_DEADLINE_EXCEEDED" { + panic(r) + } tryErr = fmt.Errorf("%v", r) } }() @@ -3438,6 +3442,13 @@ func (vm *BCVM) run(insts []bytecode.BCInstruction, env *BcEnv) any { if err != nil { panic(NewRuntimeError("TYPE_ERROR", "main", ip, inst.Op, "sleep requires number, got %T", msAny)) } + if deadline, ok := vm.ctx.Deadline(); ok && ms > 0 && (ms > int64(time.Until(deadline)/time.Millisecond)) { + code := "LIMIT_EXCEEDED" + if vm.prog.IsModelAdapter() { + code = "HFIR_DEADLINE_EXCEEDED" + } + panic(NewRuntimeError(code, "main", ip, inst.Op, "wall-clock deadline exceeded: sleep exceeds remaining budget")) + } timer := time.NewTimer(time.Duration(ms) * time.Millisecond) select { case <-timer.C: @@ -3845,7 +3856,11 @@ func (vm *BCVM) chargeBytes(n, ip int, op bytecode.Opcode) { func (vm *BCVM) checkDeadline(ip int, op bytecode.Opcode) { if vm.ctx != nil && vm.ctx.Err() != nil { - panic(NewRuntimeError("LIMIT_EXCEEDED", "main", ip, op, "wall-clock deadline exceeded")) + code := "LIMIT_EXCEEDED" + if vm.prog.IsModelAdapter() { + code = "HFIR_DEADLINE_EXCEEDED" + } + panic(NewRuntimeError(code, "main", ip, op, "wall-clock deadline exceeded")) } } @@ -4033,3 +4048,10 @@ func TextOf(v any) string { } return fmt.Sprint(v) } + +func effectiveExecutionPolicy(prog *bytecode.BCProgram, policy ExecutionPolicy) ExecutionPolicy { + if prog.IsModelAdapter() && (policy.Deadline <= 0 || policy.Deadline > time.Second) { + policy.Deadline = time.Second + } + return policy +}