Repository navigation
Compact conditionals: ver_gt/ver_ge, all/any, inline list_get/parse_json (+ benchmark rerun) - #135
Conversation
…son; benchmark rerun
howlcipher
left a comment
There was a problem hiding this comment.
Dev-lead review (COMMENT only) — head 6ee76f7be784087ac7cf4ef8bb1b56e9ea47a7d4, base f930cf6e.
Verdict: merge-ready on content. CI build is green on this head. Okabe may undraft and squash-merge.
Scope (77 files, 1 commit)
internal/ast/desugar{,_compact}.go (+test), internal/construct/construct.go (registry + governed parse_json branch), internal/bytecode/construct_drift_test.go, compact_conditionals_cli_test.go, experiments/token-efficiency-v0/ (60 .howl solutions, results, REPORT, verification_test.go, solutions_test.go comment), author_solutions.py, docs (construct_coverage, 2 journals). No VM, opcode, HFIR or backend changes. Python/JS/Go solutions: zero diff.
Hard nos
HFIR default and prod -compile-bc not flipped; #90 still Partial; no live LLM; no H1 PASS/KILL claim. ✅
Fail-closed check (the key ask): passes
hfir.VerifyProfile→construct.ScanProfileruns on the pre-desugar tree at check/build time, so it is static and nothing has executed yet. The PR does not move that call. It changes what the scan accepts.construct.scanSupportedcaseparse_json: underProfileGoverned, aListoperand goes through the ordinaryscanNode. That means:- forbidden heads (
exec,spawn,spawn_agent,llm_generate,sql_query, …) are rejected viaAllowed; - unknown heads are rejected ("no bytecode lowering");
- computed heads and dict pairs recurse into their children, so nesting at any depth is still scanned.
- forbidden heads (
- A non-List, non-SYMBOL operand (string or int literal) still hits the old violation, and the default
Scanstays strict. I found no shape that is accepted without being scanned. - Capabilities: the desugar only rewrites to
(let (__pjN <expr>) (parse_json T __pjN)), so the operand compiles through the normal lowering and its effects still show up in inferred caps and runtime gates. Desugar is shared bybytecode.go,vm.goandhowlframe.go, so all paths see the same expansion. TestGovernedInlineParseJSONFailsClosedcovers nestedexec/llm_generate/spawnafter awrite_file: there is no artifact, no side file and no exec output on eithercheckorbuild. It passes locally.
Built-in purity: passes
ver_gt/ver_ge/all/any/inline list_get are CompileTimeOnly sugar that expands into let/for/when/not/set/str_split/to_int/list_get/comparisons. They add no opcode or capability (inspect --caps is empty in the CLI test, and the closed pure-head set is in desugar_compact_test.go). The scan still descends into their arguments. I probed failure cases locally and each one fails deterministically, with no panic:
ver_gt "1.x.3" …→CONVERSION_ERROR"1.2"→CONVERSION_ERROR- a non-string argument → compile-time type error
all/anyover an int or string →TYPE_ERROR for requires a list
Benchmark: recomputed from results/tokens.json, matches REPORT and the PR body
| #134 (cl100k/o200k) | #135 | |
|---|---|---|
| HowlFrame median | 124/123 | 117/116 |
| vs Python (median per-task ratio) | 1.059/1.058 | 0.983/0.983 |
| vs JS | 1.019/0.953 | 0.959/0.896 |
| vs Go | 0.496/0.492 | 0.445/0.446 |
| Conditional mean (22 tasks) | 344.55 | 186.00 (Python 196.00) |
All 240 entries pass. verification_test.go is real: it checks for exactly 240 entries, no duplicates, every task×lang present, pass true, and the SHA-256 of each committed source.
Non-blockers
- Extra version parts are silently ignored.
(ver_gt "1.2.3.9" "1.2.3")returnsfalse. REPORT calls the comparison "strict x.y.z" and only documents that fewer parts fail. Either reject a 4th part or document the behavior. - Leading zeros:
ver_ge "01.2.3" "1.2.3"is true. That's fine for numeric comparison, but it's worth a line in the docs. - Gensym names are global.
compactGensymis a process-global atomic counter, so temp names depend on how many compiles came before in the same process (LSP, tests, batch builds). If HFBC bytes or receipt hashes include local names, in-process rebuilds won't be byte-identical. A per-Desugar counter would fix this. - The "profile check moved earlier" wording overstates the change. The call site is unchanged; the change is that the pre-desugar scan now descends into the operand. The REPORT text is accurate, so only the summary wording needs adjusting.
- Optional extra test cases for the fail-closed test: an unknown head inside the operand, a forbidden construct two levels deep (e.g.
(parse_json T (str_trim (exec …)))), and aver_gtwith an effectful argument under governed. - Threat to validity: the new built-ins were shaped by this benchmark, which REPORT acknowledges. Re-measuring on held-out tasks would strengthen the claim of beating Python.
…r, sum/min/max, %, regex_match, str_slice, pad_left, text/CSV/Markdown output; no directory listing exists); tuned #135 built-ins stay out; pin rendered blind prompts; regenerate pins
Compact-conditional built-ins, plus a token-efficiency-v0 rerun. Measure only. Based on main
f930cf6e.Language changes
All four are
ast.Desugarsugar that expands into constructs that already lower. They add no opcode, capability or effect.(ver_gt a b)and(ver_ge a b): strict numeric x.y.z comparison.(all x list pred)and(any x list pred).list_getaccepts any expression as its list. Literal(list …)operands are left untouched so wasm keeps working.(parse_json T <expr>)under governed:construct.Scanstays strict.The four new names are registered as
CompileTimeOnlyand added to the bytecode drift list.construct_coverage.mdis regenerated.Guards
internal/ast/desugar_compact_test.go: expansions use only a closed set of pure heads, and no sugar head survives desugar.TestCompactConditionalBuiltinsArePureAndCorrect: a governed build whereinspect --capsreports no capabilities, plus a truth table that includes empty lists, equal versions and 1.10.0 vs 1.9.9.TestGovernedInlineParseJSONFailsClosed:exec,llm_generateorspawnis placed inside an inlineparse_jsonafter awrite_file. Governedcheckandbuildmust fail withPROFILE_FORBIDDEN_CONSTRUCT, and no artifact, side file or exec output may exist afterwards.verification_test.gochecks 60×4 entries, all passing, each with a matching SHA-256.Benchmark (cl100k / o200k)
Python, JS and Go solutions are byte-identical.
Remaining hot spots:
dictpairs and the escaped JSON literal.Docs
docs/journals/2026-10-12_compact_conditionals.md.doToken density sugar: beat Python on Task C write-cost tokens #117,read_text/linesAdd (lines)/(read_text) sugar; beat Python on Task B tokens #118, multi-bindingletProve real-app token wins without losing functionality #120 andwhenDogfood fetch/template/stamp; map_has, when/unless, n-ary and/or #124. It was not "Dogfood fetch/template/stamp; map_has, when/unless, n-ary and/or #124–Dogfood case/trim CONVERT; lower/upper/trim/icontains apps #130" or "Dogfood join/uniq/echo; http_status/http_res/not sugar #128–Dogfood case/trim CONVERT; lower/upper/trim/icontains apps #130". Fixed in the report and the Token benchmark rerun on current shorthand (measure only) #134 journal.Validation (local, unsandboxed)
go build ./... && go vet ./... && go test ./...: exit 0.go test ./experiments/...: passes.run.sh: 240/240 pass.No production
-compile-bcor HFIR change, #90 stays Partial, and there were no live LLM or H1 runs.