Skip to content

Compact conditionals: ver_gt/ver_ge, all/any, inline list_get/parse_json (+ benchmark rerun) - #135

Merged
howlcipher merged 1 commit into
mainfrom
compact-conditionals
Oct 12, 2026
Merged

howlcipher merged 1 commit into
mainfrom
compact-conditionals

Conversation

@howlcipher

Copy link
Copy Markdown
Owner

Compact-conditional built-ins, plus a token-efficiency-v0 rerun. Measure only. Based on main f930cf6e.

Language changes

All four are ast.Desugar sugar that expands into constructs that already lower. They add no opcode, capability or effect.

  1. (ver_gt a b) and (ver_ge a b): strict numeric x.y.z comparison.
  2. (all x list pred) and (any x list pred).
  3. list_get accepts any expression as its list. Literal (list …) operands are left untouched so wasm keeps working.
  4. Inline (parse_json T <expr>) under governed:
    • The pre-desugar governed profile scan now descends into the operand instead of rejecting the shape.
    • Forbidden constructs inside the operand are still rejected.
    • The post-desugar scans and the construct gate are unchanged, and the default construct.Scan stays strict.

The four new names are registered as CompileTimeOnly and added to the bytecode drift list. construct_coverage.md is regenerated.

Guards

  • internal/ast/desugar_compact_test.go: expansions use only a closed set of pure heads, and no sugar head survives desugar.
  • TestCompactConditionalBuiltinsArePureAndCorrect: a governed build where inspect --caps reports no capabilities, plus a truth table that includes empty lists, equal versions and 1.10.0 vs 1.9.9.
  • TestGovernedInlineParseJSONFailsClosed: exec, llm_generate or spawn is placed inside an inline parse_json after a write_file. Governed check and build must fail with PROFILE_FORBIDDEN_CONSTRUCT, and no artifact, side file or exec output may exist afterwards.
  • Motoko follow-up: verification_test.go checks 60×4 entries, all passing, each with a matching SHA-256.

Benchmark (cl100k / o200k)

Python, JS and Go solutions are byte-identical.

Measure Before (#134) After
HowlFrame median 124/123 117/116 (Python 119/118, JS 122/129)
vs Python 1.059/1.058 0.983/0.983: larger on 0 tasks, 16 ties
vs JS 1.019/0.953 0.959/0.896: larger on 0 tasks
vs Go 0.496/0.492 0.445/0.446
Conditional-task mean 344.55 186.00 (Python 196.00)

Remaining hot spots:

  1. The version KV extraction operands, about 35 tokens.
  2. The filter loop, about 27 tokens against Python's 17-token comprehension. This is why the write-record tasks only tie Python.
  3. dict pairs and the escaped JSON literal.

Docs

Validation (local, unsandboxed)

  • go build ./... && go vet ./... && go test ./...: exit 0.
  • go test ./experiments/...: passes.
  • run.sh: 240/240 pass.

No production -compile-bc or HFIR change, #90 stays Partial, and there were no live LLM or H1 runs.

@howlcipher howlcipher left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dev-lead review (COMMENT only) — head 6ee76f7be784087ac7cf4ef8bb1b56e9ea47a7d4, base f930cf6e.

Verdict: merge-ready on content. CI build is green on this head. Okabe may undraft and squash-merge.

Scope (77 files, 1 commit)

internal/ast/desugar{,_compact}.go (+test), internal/construct/construct.go (registry + governed parse_json branch), internal/bytecode/construct_drift_test.go, compact_conditionals_cli_test.go, experiments/token-efficiency-v0/ (60 .howl solutions, results, REPORT, verification_test.go, solutions_test.go comment), author_solutions.py, docs (construct_coverage, 2 journals). No VM, opcode, HFIR or backend changes. Python/JS/Go solutions: zero diff.

Hard nos

HFIR default and prod -compile-bc not flipped; #90 still Partial; no live LLM; no H1 PASS/KILL claim. ✅

Fail-closed check (the key ask): passes

  • hfir.VerifyProfile → construct.ScanProfile runs on the pre-desugar tree at check/build time, so it is static and nothing has executed yet. The PR does not move that call. It changes what the scan accepts.
  • construct.scanSupported case parse_json: under ProfileGoverned, a List operand goes through the ordinary scanNode. That means:
    • forbidden heads (exec, spawn, spawn_agent, llm_generate, sql_query, …) are rejected via Allowed;
    • unknown heads are rejected ("no bytecode lowering");
    • computed heads and dict pairs recurse into their children, so nesting at any depth is still scanned.
  • A non-List, non-SYMBOL operand (string or int literal) still hits the old violation, and the default Scan stays strict. I found no shape that is accepted without being scanned.
  • Capabilities: the desugar only rewrites to (let (__pjN <expr>) (parse_json T __pjN)), so the operand compiles through the normal lowering and its effects still show up in inferred caps and runtime gates. Desugar is shared by bytecode.go, vm.go and howlframe.go, so all paths see the same expansion.
  • TestGovernedInlineParseJSONFailsClosed covers nested exec/llm_generate/spawn after a write_file: there is no artifact, no side file and no exec output on either check or build. It passes locally.

Built-in purity: passes

ver_gt/ver_ge/all/any/inline list_get are CompileTimeOnly sugar that expands into let/for/when/not/set/str_split/to_int/list_get/comparisons. They add no opcode or capability (inspect --caps is empty in the CLI test, and the closed pure-head set is in desugar_compact_test.go). The scan still descends into their arguments. I probed failure cases locally and each one fails deterministically, with no panic:

  • ver_gt "1.x.3" … → CONVERSION_ERROR
  • "1.2" → CONVERSION_ERROR
  • a non-string argument → compile-time type error
  • all/any over an int or string → TYPE_ERROR for requires a list

Benchmark: recomputed from results/tokens.json, matches REPORT and the PR body

#134 (cl100k/o200k) #135
HowlFrame median 124/123 117/116
vs Python (median per-task ratio) 1.059/1.058 0.983/0.983
vs JS 1.019/0.953 0.959/0.896
vs Go 0.496/0.492 0.445/0.446
Conditional mean (22 tasks) 344.55 186.00 (Python 196.00)

All 240 entries pass. verification_test.go is real: it checks for exactly 240 entries, no duplicates, every task×lang present, pass true, and the SHA-256 of each committed source.

Non-blockers

  1. Extra version parts are silently ignored. (ver_gt "1.2.3.9" "1.2.3") returns false. REPORT calls the comparison "strict x.y.z" and only documents that fewer parts fail. Either reject a 4th part or document the behavior.
  2. Leading zeros: ver_ge "01.2.3" "1.2.3" is true. That's fine for numeric comparison, but it's worth a line in the docs.
  3. Gensym names are global. compactGensym is a process-global atomic counter, so temp names depend on how many compiles came before in the same process (LSP, tests, batch builds). If HFBC bytes or receipt hashes include local names, in-process rebuilds won't be byte-identical. A per-Desugar counter would fix this.
  4. The "profile check moved earlier" wording overstates the change. The call site is unchanged; the change is that the pre-desugar scan now descends into the operand. The REPORT text is accurate, so only the summary wording needs adjusting.
  5. Optional extra test cases for the fail-closed test: an unknown head inside the operand, a forbidden construct two levels deep (e.g. (parse_json T (str_trim (exec …)))), and a ver_gt with an effectful argument under governed.
  6. Threat to validity: the new built-ins were shaped by this benchmark, which REPORT acknowledges. Re-measuring on held-out tasks would strengthen the claim of beating Python.

@howlcipher
howlcipher marked this pull request as ready for review October 12, 2026 15:06
@howlcipher
howlcipher merged commit 24f6a10 into main Oct 12, 2026
1 check passed
howlcipher added a commit that referenced this pull request Oct 13, 2026
…r, sum/min/max, %, regex_match, str_slice, pad_left, text/CSV/Markdown output; no directory listing exists); tuned #135 built-ins stay out; pin rendered blind prompts; regenerate pins
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant