diff --git a/README.md b/README.md index 905188a..2019db3 100644 --- a/README.md +++ b/README.md @@ -60,7 +60,9 @@ out, err := tmpl.Render(ctx, env) The text outside `${...}` is just text. The expression inside can use the same selectors, functions, literals, and higher-order forms as any other compiled -expression. +expression. `nil` renders as the empty string; maps, slices, arrays, and +structs render as compact JSON (`${config}` produces `{"retries":3}`, not +`map[retries:3]`). ## Using the Go API @@ -90,9 +92,11 @@ safe to share between goroutines. Compile at startup. Run per request. No functions are registered by default, so the surface area is exactly as wide as you want it. `WithBuiltins()` opts you into a small standard set (`len`, -`contains`, `has`, `keys`, `upper`, `lower`, `int`, `float`, `string`, `bool`, -`sprintf`). `WithFunctions` lets you register any Go function as a callable -identifier: +`contains`, `has`, `keys`, `entries`, `upper`, `lower`, `int`, `float`, +`string`, `bool`, `sprintf`). `entries(m)` returns the sorted key-value pairs +of a string-keyed map as `[{"key":k,"value":v}, ...]`, making maps iterable +through higher-order forms. `WithFunctions` lets you register any Go function +as a callable identifier: ```go p, err := expr.Compile(`greet(upper(name))`, expr.WithFunctions(map[string]any{ @@ -105,8 +109,8 @@ Mix and match, or skip the builtins entirely and expose only the handful that make sense for your sandbox. Opt-in groups — `expr.MathFuncs()` (`min`, `max`, `abs`, `floor`, `ceil`, `round`), `expr.StringFuncs()` (`trim`, `split`, `join`, `replace`, `startsWith`, `endsWith`), and `expr.CollectionFuncs()` -(`first`, `last`, `sum`, `slice`) — add the usual helpers via `WithFunctions` -without widening the default set. +(`first`, `last`, `sum`, `slice`, `sort`, `reverse`) — add the usual helpers +via `WithFunctions` without widening the default set. ## What the environment can be @@ -143,22 +147,33 @@ are left alone, so nothing you already had stops working. ## Higher-order forms -A small set of always-available forms for working with lists: `map`, `filter`, -`any`, `all`, `find`, `count`. Inside the second argument, `it` is the current -element and `index` is its position: +A set of always-available forms for working with lists: `map`, `filter`, +`flatMap`, `any`, `all`, `find`, `count`, `sortBy`. Inside the body, `it` is +the current element and `index` is its position: ```go p, err := expr.Compile(`filter(users, it.age >= 18 && index < 10)`) ``` -The predicate is re-evaluated per element, so they compose naturally: -`any(orders, count(it.items, it.price > 100) > 0)`. Two more special forms -use laziness for control flow instead of iteration: `try(value, default)` -falls back when `value` errors, and `if(cond, then, else)` evaluates only the -branch the condition selects — so `if(n != 0, total/n, 0)` can't divide by -zero. These forms are always registered (no `WithBuiltins` needed), but you -can shadow any of them by registering a function or env value of the same -name. +Every iterating form also accepts a three-argument shape that names the element +explicitly, which makes nested forms readable and lets you reference an outer +element from inside an inner body: + +```go +// Named bindings: r is the review, c is the comment. +map(reviews, r, map(r.comments, c, r.author + ": " + c)) +``` + +`flatMap` works like `map` but splices list body results element-by-element +into the output, which flattens one level of nesting. `sortBy` evaluates a key +expression per element and returns a stable-sorted copy of the list. + +Two more special forms use laziness for control flow instead of iteration: +`try(value, default)` falls back when `value` errors, and `if(cond, then, +else)` evaluates only the branch the condition selects, so `if(n != 0, total/n, +0)` can't divide by zero. These forms are always registered (no `WithBuiltins` +needed), but you can shadow any of them by registering a function or env value +of the same name. ## What it isn't diff --git a/builtin_groups.go b/builtin_groups.go index 44f0e13..6808534 100644 --- a/builtin_groups.go +++ b/builtin_groups.go @@ -5,6 +5,7 @@ import ( "fmt" "math" "reflect" + "sort" "strings" ) @@ -63,12 +64,17 @@ func StringFuncs() map[string]any { // slice(xs, i, j) elements [i, j) of a list, or the rune range // of a string; negative indices count from the // end and out-of-range bounds clamp +// sort(xs) ascending copy; all numbers (numeric order) or +// all strings (lexicographic); mixed types error +// reverse(xs) reversed copy; never mutates input func CollectionFuncs() map[string]any { return map[string]any{ - "first": Func(nativeFirst), - "last": Func(nativeLast), - "sum": Func(nativeSum), - "slice": Func(nativeSlice), + "first": Func(nativeFirst), + "last": Func(nativeLast), + "sum": Func(nativeSum), + "slice": Func(nativeSlice), + "sort": Func(nativeSort), + "reverse": Func(nativeReverse), } } @@ -399,3 +405,110 @@ func resolveIndex(i int64, n int) int { } return int(i) } + +func nativeSort(_ context.Context, args []any) (any, error) { + if err := checkArity("sort", 1, len(args)); err != nil { + return nil, err + } + return builtinSort(args[0]) +} + +// builtinSort returns a sorted copy of a list. All elements must be +// either numbers (any int/float mix, compared with the same rules as +// the < operator) or all strings (lexicographic). The sort is stable +// and reorders the original elements without converting them: ints +// stay ints, floats stay floats. Mixed or non-comparable element +// types produce an ErrEvaluate. An empty or nil input returns an +// empty []any. +func builtinSort(v any) ([]any, error) { + if v == nil { + return []any{}, nil + } + rv := reflect.ValueOf(v) + if rv.Kind() != reflect.Slice && rv.Kind() != reflect.Array { + return nil, fmt.Errorf("%w: sort: expected list, got %T", ErrEvaluate, v) + } + n := rv.Len() + out := make([]any, n) + for i := 0; i < n; i++ { + out[i] = rv.Index(i).Interface() + } + less, err := scalarLessFunc("sort", out) + if err != nil { + return nil, err + } + sort.SliceStable(out, less) + return out, nil +} + +// scalarLessFunc validates that every value in vals is a number (any +// int/float mix) or that every value is a string, and returns the +// matching index-based less function. The mode is chosen by the first +// value; the error names the first value that does not fit. Shared by +// sort and sortBy so the two agree on comparison semantics. +func scalarLessFunc(name string, vals []any) (func(i, j int) bool, error) { + if len(vals) == 0 { + return func(i, j int) bool { return false }, nil + } + if _, ok := toFloat64(vals[0]); ok { + for i, v := range vals { + if _, ok := toFloat64(v); !ok { + return nil, fmt.Errorf("%w: %s: element %d is %T, not a number", ErrEvaluate, name, i, v) + } + } + return func(i, j int) bool { return numericLess(vals[i], vals[j]) }, nil + } + if _, ok := asString(vals[0]); ok { + for i, v := range vals { + if _, ok := asString(v); !ok { + return nil, fmt.Errorf("%w: %s: element %d is %T, not a string", ErrEvaluate, name, i, v) + } + } + return func(i, j int) bool { + a, _ := asString(vals[i]) + b, _ := asString(vals[j]) + return a < b + }, nil + } + return nil, fmt.Errorf("%w: %s: elements must be all numbers or all strings, got %T", + ErrEvaluate, name, vals[0]) +} + +// numericLess compares two numbers with the same rules as the < +// operator: both integral values compare as int64, any other mix +// compares as float64. +func numericLess(a, b any) bool { + if ai, ok := toInt64(a); ok { + if bi, ok := toInt64(b); ok { + return ai < bi + } + } + af, _ := toFloat64(a) + bf, _ := toFloat64(b) + return af < bf +} + +func nativeReverse(_ context.Context, args []any) (any, error) { + if err := checkArity("reverse", 1, len(args)); err != nil { + return nil, err + } + return builtinReverse(args[0]) +} + +// builtinReverse returns a reversed copy of a list. It never mutates +// the input. nil and empty lists return an empty []any. +func builtinReverse(v any) ([]any, error) { + if v == nil { + return []any{}, nil + } + rv := reflect.ValueOf(v) + if rv.Kind() != reflect.Slice && rv.Kind() != reflect.Array { + return nil, fmt.Errorf("%w: reverse: expected list, got %T", ErrEvaluate, v) + } + n := rv.Len() + out := make([]any, n) + for i := 0; i < n; i++ { + out[i] = rv.Index(n - 1 - i).Interface() + } + return out, nil +} diff --git a/builtins.go b/builtins.go index ddc6b64..bab87fc 100644 --- a/builtins.go +++ b/builtins.go @@ -29,6 +29,8 @@ import ( // key presence for string-keyed maps // has(m, k) true if map m has key k; errors if m is not a map // keys(m) sorted string keys of a map +// entries(m) sorted key-value pairs of a string-keyed map; each +// element is map[string]any{"key": k, "value": v} // lower(s), upper(s) case conversion // sprintf(fmt, ...) fmt.Sprintf-style formatting with cycle guards // @@ -46,6 +48,7 @@ func Builtins() map[string]any { "contains": Func(nativeContains), "has": Func(nativeHas), "keys": Func(nativeKeys), + "entries": Func(nativeEntries), "lower": Func(nativeLower), "upper": Func(nativeUpper), "sprintf": Func(nativeSprintf), @@ -116,6 +119,13 @@ func nativeKeys(_ context.Context, args []any) (any, error) { return builtinKeys(args[0]) } +func nativeEntries(_ context.Context, args []any) (any, error) { + if err := checkArity("entries", 1, len(args)); err != nil { + return nil, err + } + return builtinEntries(args[0]) +} + func nativeLower(_ context.Context, args []any) (any, error) { if err := checkArity("lower", 1, len(args)); err != nil { return nil, err @@ -307,3 +317,29 @@ func builtinKeys(m any) ([]any, error) { } return out, nil } + +// builtinEntries returns the key-value pairs of a string-keyed map as a +// []any, sorted by key for determinism. Each element is a +// map[string]any{"key": k, "value": v}, mirroring the sort and key-type +// rules of builtinKeys. +func builtinEntries(m any) ([]any, error) { + if m == nil { + return nil, nil + } + rv := reflect.ValueOf(m) + if rv.Kind() != reflect.Map || rv.Type().Key().Kind() != reflect.String { + return nil, fmt.Errorf("%w: entries: expected map with string keys, got %T", ErrEvaluate, m) + } + mapKeys := rv.MapKeys() + strs := make([]string, len(mapKeys)) + for i, k := range mapKeys { + strs[i] = k.String() + } + sort.Strings(strs) + out := make([]any, len(strs)) + for i, s := range strs { + val := rv.MapIndex(mapStringKey(rv.Type().Key(), s)).Interface() + out[i] = map[string]any{"key": s, "value": val} + } + return out, nil +} diff --git a/builtins_entries_sort_test.go b/builtins_entries_sort_test.go new file mode 100644 index 0000000..b6ad2ea --- /dev/null +++ b/builtins_entries_sort_test.go @@ -0,0 +1,286 @@ +package expr + +import ( + "testing" + + "github.com/deepnoodle-ai/expr/internal/require" +) + +// --------------------------------------------------------------------------- +// entries() +// --------------------------------------------------------------------------- + +func TestEntries_EmptyMap(t *testing.T) { + got, err := evalExpr(t.Context(), `entries(m)`, map[string]any{"m": map[string]any{}}, WithBuiltins()) + require.NoError(t, err) + require.Equal(t, []any{}, got) +} + +func TestEntries_NilMap(t *testing.T) { + got, err := evalExpr(t.Context(), `entries(m)`, map[string]any{"m": nil}, WithBuiltins()) + require.NoError(t, err) + require.Nil(t, got) +} + +func TestEntries_SingleKey(t *testing.T) { + m := map[string]any{"x": int64(42)} + got, err := evalExpr(t.Context(), `entries(m)`, map[string]any{"m": m}, WithBuiltins()) + require.NoError(t, err) + want := []any{ + map[string]any{"key": "x", "value": int64(42)}, + } + require.Equal(t, want, got) +} + +// entries must sort by key for determinism regardless of map iteration order. +func TestEntries_MultiKeySorted(t *testing.T) { + m := map[string]any{"c": int64(3), "a": int64(1), "b": int64(2)} + got, err := evalExpr(t.Context(), `entries(m)`, map[string]any{"m": m}, WithBuiltins()) + require.NoError(t, err) + want := []any{ + map[string]any{"key": "a", "value": int64(1)}, + map[string]any{"key": "b", "value": int64(2)}, + map[string]any{"key": "c", "value": int64(3)}, + } + require.Equal(t, want, got) +} + +// entries must match the sort order produced by keys() for the same map. +func TestEntries_SortMatchesKeys(t *testing.T) { + m := map[string]any{"zebra": "z", "apple": "a", "mango": "m"} + env := map[string]any{"m": m} + + keysGot, err := evalExpr(t.Context(), `keys(m)`, env, WithBuiltins()) + require.NoError(t, err) + + entriesGot, err := evalExpr(t.Context(), `entries(m)`, env, WithBuiltins()) + require.NoError(t, err) + + keySlice := keysGot.([]any) + entrySlice := entriesGot.([]any) + require.Equal(t, len(keySlice), len(entrySlice)) + for i, k := range keySlice { + entry := entrySlice[i].(map[string]any) + require.Equal(t, k, entry["key"], "position %d key mismatch", i) + } +} + +// Non-string-keyed map must produce an ErrEvaluate, same class as keys(). +func TestEntries_NonStringKeyError(t *testing.T) { + _, err := evalExpr(t.Context(), `entries(m)`, map[string]any{"m": map[int]any{1: "x"}}, WithBuiltins()) + require.ErrorIs(t, err, ErrEvaluate) + require.Contains(t, err.Error(), "entries") +} + +// entries arity check. +func TestEntries_ArityError(t *testing.T) { + _, err := evalExpr(t.Context(), `entries(m, m)`, map[string]any{"m": map[string]any{}}, WithBuiltins()) + require.ErrorIs(t, err, ErrEvaluate) +} + +// Compose entries with filter: keep only entries whose value > 1. +func TestEntries_ComposeWithFilter(t *testing.T) { + m := map[string]any{"a": int64(1), "b": int64(2), "c": int64(3)} + env := map[string]any{"m": m} + // filter(entries(m), it.value > 1) returns entries with value 2 and 3. + got, err := evalExpr(t.Context(), + `filter(entries(m), it.value > 1)`, + env, + WithBuiltins(), + ) + require.NoError(t, err) + want := []any{ + map[string]any{"key": "b", "value": int64(2)}, + map[string]any{"key": "c", "value": int64(3)}, + } + require.Equal(t, want, got) +} + +// Compose entries with map form to extract keys. +func TestEntries_ComposeWithMapForm(t *testing.T) { + m := map[string]any{"b": int64(2), "a": int64(1)} + env := map[string]any{"m": m} + // map(entries(m), it.key) should equal keys(m) + got, err := evalExpr(t.Context(), `map(entries(m), it.key)`, env, WithBuiltins()) + require.NoError(t, err) + require.Equal(t, []any{"a", "b"}, got) +} + +// --------------------------------------------------------------------------- +// sort() +// --------------------------------------------------------------------------- + +func TestSort_Ints(t *testing.T) { + env := map[string]any{"xs": []any{int64(3), int64(1), int64(2)}} + got, err := evalExpr(t.Context(), `sort(xs)`, env, WithFunctions(CollectionFuncs())) + require.NoError(t, err) + require.Equal(t, []any{int64(1), int64(2), int64(3)}, got) +} + +func TestSort_Floats(t *testing.T) { + env := map[string]any{"xs": []any{3.5, 1.1, 2.2}} + got, err := evalExpr(t.Context(), `sort(xs)`, env, WithFunctions(CollectionFuncs())) + require.NoError(t, err) + // All floats, all whole-or-fractional: result stays float64. + sl := got.([]any) + require.Equal(t, 3, len(sl)) + require.Equal(t, 1.1, sl[0]) + require.Equal(t, 2.2, sl[1]) + require.Equal(t, 3.5, sl[2]) +} + +func TestSort_MixedIntFloat(t *testing.T) { + // int64 and float64 are both numeric; sort numerically. + env := map[string]any{"xs": []any{int64(3), 1.5, int64(2)}} + got, err := evalExpr(t.Context(), `sort(xs)`, env, WithFunctions(CollectionFuncs())) + require.NoError(t, err) + sl := got.([]any) + require.Equal(t, 3, len(sl)) + // 1.5 first, then 2, then 3 + f0, ok0 := toFloat64(sl[0]) + f1, ok1 := toFloat64(sl[1]) + f2, ok2 := toFloat64(sl[2]) + require.True(t, ok0 && ok1 && ok2) + require.Equal(t, 1.5, f0) + require.Equal(t, float64(2), f1) + require.Equal(t, float64(3), f2) +} + +func TestSort_Strings(t *testing.T) { + env := map[string]any{"xs": []any{"banana", "apple", "cherry"}} + got, err := evalExpr(t.Context(), `sort(xs)`, env, WithFunctions(CollectionFuncs())) + require.NoError(t, err) + require.Equal(t, []any{"apple", "banana", "cherry"}, got) +} + +func TestSort_Empty(t *testing.T) { + got, err := evalExpr(t.Context(), `sort([])`, nil, WithFunctions(CollectionFuncs())) + require.NoError(t, err) + require.Equal(t, []any{}, got) +} + +func TestSort_Nil(t *testing.T) { + got, err := evalExpr(t.Context(), `sort(xs)`, map[string]any{"xs": nil}, WithFunctions(CollectionFuncs())) + require.NoError(t, err) + require.Equal(t, []any{}, got) +} + +func TestSort_SingleElement(t *testing.T) { + got, err := evalExpr(t.Context(), `sort([42])`, nil, WithFunctions(CollectionFuncs())) + require.NoError(t, err) + require.Equal(t, []any{int64(42)}, got) +} + +func TestSort_MixedTypeError(t *testing.T) { + env := map[string]any{"xs": []any{int64(1), "two", int64(3)}} + _, err := evalExpr(t.Context(), `sort(xs)`, env, WithFunctions(CollectionFuncs())) + require.ErrorIs(t, err, ErrEvaluate) + require.Contains(t, err.Error(), "sort") +} + +func TestSort_BoolError(t *testing.T) { + env := map[string]any{"xs": []any{true, false}} + _, err := evalExpr(t.Context(), `sort(xs)`, env, WithFunctions(CollectionFuncs())) + require.ErrorIs(t, err, ErrEvaluate) +} + +func TestSort_NilElementError(t *testing.T) { + env := map[string]any{"xs": []any{int64(1), nil, int64(3)}} + _, err := evalExpr(t.Context(), `sort(xs)`, env, WithFunctions(CollectionFuncs())) + require.ErrorIs(t, err, ErrEvaluate) +} + +func TestSort_NotAListError(t *testing.T) { + _, err := evalExpr(t.Context(), `sort(42)`, nil, WithFunctions(CollectionFuncs())) + require.ErrorIs(t, err, ErrEvaluate) + require.Contains(t, err.Error(), "sort") +} + +// sort must not mutate the input slice. +func TestSort_InputNotMutated(t *testing.T) { + original := []any{int64(3), int64(1), int64(2)} + // make a copy to check against + snapshot := []any{int64(3), int64(1), int64(2)} + env := map[string]any{"xs": original} + _, err := evalExpr(t.Context(), `sort(xs)`, env, WithFunctions(CollectionFuncs())) + require.NoError(t, err) + require.Equal(t, snapshot, original) +} + +// Expressions with inline literals work too. +func TestSort_InlineLiteral(t *testing.T) { + got, err := evalExpr(t.Context(), `sort([3, 1, 2])`, nil, WithFunctions(CollectionFuncs())) + require.NoError(t, err) + require.Equal(t, []any{int64(1), int64(2), int64(3)}, got) +} + +// --------------------------------------------------------------------------- +// reverse() +// --------------------------------------------------------------------------- + +func TestReverse_List(t *testing.T) { + env := map[string]any{"xs": []any{int64(1), int64(2), int64(3)}} + got, err := evalExpr(t.Context(), `reverse(xs)`, env, WithFunctions(CollectionFuncs())) + require.NoError(t, err) + require.Equal(t, []any{int64(3), int64(2), int64(1)}, got) +} + +func TestReverse_Strings(t *testing.T) { + env := map[string]any{"xs": []any{"a", "b", "c"}} + got, err := evalExpr(t.Context(), `reverse(xs)`, env, WithFunctions(CollectionFuncs())) + require.NoError(t, err) + require.Equal(t, []any{"c", "b", "a"}, got) +} + +func TestReverse_Empty(t *testing.T) { + got, err := evalExpr(t.Context(), `reverse([])`, nil, WithFunctions(CollectionFuncs())) + require.NoError(t, err) + require.Equal(t, []any{}, got) +} + +func TestReverse_Nil(t *testing.T) { + got, err := evalExpr(t.Context(), `reverse(xs)`, map[string]any{"xs": nil}, WithFunctions(CollectionFuncs())) + require.NoError(t, err) + require.Equal(t, []any{}, got) +} + +func TestReverse_SingleElement(t *testing.T) { + got, err := evalExpr(t.Context(), `reverse([99])`, nil, WithFunctions(CollectionFuncs())) + require.NoError(t, err) + require.Equal(t, []any{int64(99)}, got) +} + +func TestReverse_NotAListError(t *testing.T) { + _, err := evalExpr(t.Context(), `reverse("hello")`, nil, WithFunctions(CollectionFuncs())) + require.ErrorIs(t, err, ErrEvaluate) + require.Contains(t, err.Error(), "reverse") +} + +// reverse must not mutate the input slice. +func TestReverse_InputNotMutated(t *testing.T) { + original := []any{int64(1), int64(2), int64(3)} + snapshot := []any{int64(1), int64(2), int64(3)} + env := map[string]any{"xs": original} + _, err := evalExpr(t.Context(), `reverse(xs)`, env, WithFunctions(CollectionFuncs())) + require.NoError(t, err) + require.Equal(t, snapshot, original) +} + +// Compose sort + reverse for descending order. +func TestSortReverse_Descending(t *testing.T) { + env := map[string]any{"xs": []any{int64(3), int64(1), int64(2)}} + got, err := evalExpr(t.Context(), `reverse(sort(xs))`, env, WithFunctions(CollectionFuncs())) + require.NoError(t, err) + require.Equal(t, []any{int64(3), int64(2), int64(1)}, got) +} + +// sort and reverse must not appear in the default Builtins set. +func TestSortReverse_NotInDefaultBuiltins(t *testing.T) { + _, err := evalExpr(t.Context(), `sort([1, 2])`, nil, WithBuiltins()) + require.ErrorIs(t, err, ErrEvaluate) + require.Contains(t, err.Error(), "unknown function") + + _, err = evalExpr(t.Context(), `reverse([1, 2])`, nil, WithBuiltins()) + require.ErrorIs(t, err, ErrEvaluate) + require.Contains(t, err.Error(), "unknown function") +} diff --git a/docs/guides/examples.md b/docs/guides/examples.md index 69104e8..452ec4c 100644 --- a/docs/guides/examples.md +++ b/docs/guides/examples.md @@ -202,11 +202,17 @@ map[string]any{ } ``` -Caveat worth knowing: inside a nested higher-order form, `it` and -`index` always refer to the **innermost** form's current element. -There is no `let` or outer-binding. If you need to reference both -the outer element and inner element in the same predicate, stop -nesting and do the join in Go, or register a helper function. +Named bindings let you reference the outer element by name from inside +an inner body: + +``` +filter(entries(scores), e, e.value >= 80) +``` + +Caveat for the two-arg form: inside a nested two-arg higher-order form, +`it` and `index` always refer to the **innermost** form's current +element. The outer `it` is shadowed. Use named bindings to keep both +visible. --- @@ -238,11 +244,11 @@ map[string]any{ } ``` -The `${...}` result is stringified via `fmt.Sprintf("%v", ...)`, so a -`[]any` of strings prints as a Go slice. For real templating of lists, -either build the final string with `sprintf` in a single expression, or -join the list in the host program and interpolate the joined string back -in through the env. +Maps, slices, arrays, and structs render as compact JSON inside `${...}`: +`${files}` produces `["a.go","b.go"]`, not `[a.go b.go]`. For +variable-length lists rendered as human-readable text, register a `join` +helper and call it from the expression, or join in Go and pass the result +through the env. --- @@ -345,33 +351,49 @@ a redeploy. That's the entire pitch for an embedded expression language. --- -## 8. Extracting + sorting via a registered function +## 8. Extracting + sorting -Higher-order forms don't include `sort`, on purpose — sorting needs -stable comparators and expr stays out of that business. Register a Go -function instead: +`sortBy` is a built-in special form. It evaluates a key expression per +element and returns a stable-sorted copy of the list. Combined with +`filter` and a registered `take`: ```go take( sortBy( filter(users, it.active), - "age", + it.age, + ), + 3, +) +``` + +Or with the named-binding form for clarity: + +```go +take( + sortBy( + filter(users, u, u.active), + u, + u.age, ), 3, ) ``` -Host-side registration: +Host-side `take` registration: ```go expr.WithFunctions(map[string]any{ - "sortBy": func(xs []any, key string) []any { ... }, - "take": func(xs []any, n int) []any { ... }, + "take": func(xs []any, n int) []any { ... }, }) ``` -The philosophy: if expr doesn't have it, register a Go function for it. -Don't fight the language. +`sortBy` keys must be all numbers or all strings. For descending order, +compose with `reverse` from `CollectionFuncs`: + +```go +reverse(sortBy(users, u, u.age)) +``` --- @@ -444,3 +466,91 @@ p, err := expr.Compile(src, With an empty `prices` list, `avg_price` is `0` rather than a division-by-zero error — the untaken branch never runs. + +--- + +## 11. Named bindings and flatMap + +Named element bindings let you refer to the outer element by name from +inside a nested form body. `flatMap` flattens one level of nesting. + +```go +// Extract all order IDs from all users using flatMap with a named binding. +flatMap(users, u, u.orders) +``` + +Env: + +```go +map[string]any{ + "users": []any{ + map[string]any{"orders": []any{int64(1), int64(2)}}, + map[string]any{"orders": []any{int64(3)}}, + }, +} +``` + +Result: `[]any{1, 2, 3}`. + +Nested named forms — the outer `r` stays visible inside the inner body +because the inner named form does not bind `it`: + +```go +map(reviews, r, join(map(r.comments, c, r.author + ": " + c), "; ")) +``` + +Env: + +```go +map[string]any{ + "reviews": []any{ + map[string]any{"author": "ann", "comments": []any{"good", "clear"}}, + map[string]any{"author": "bob", "comments": []any{"ok"}}, + }, +} +``` + +Result (requires `WithFunctions(expr.StringFuncs())`): +`[]any{"ann: good; ann: clear", "bob: ok"}`. + +--- + +## 12. entries, sort, and reverse + +`entries(m)` makes maps iterable through higher-order forms. `sort` and +`reverse` (from `CollectionFuncs`) sort and reverse lists. + +```go +// Format all response headers as "key: value", sorted by key. +map(entries(headers), e, sprintf("%s: %s", e.key, e.value)) +``` + +Env: + +```go +map[string]any{ + "headers": map[string]any{ + "content-type": "application/json", + "x-request-id": "abc123", + }, +} +``` + +Result: `[]any{"content-type: application/json", "x-request-id: abc123"}`. + +```go +// Keep only entries whose value exceeds a threshold. +filter(entries(scores), e, e.value > 80) +``` + +`sort` and `reverse` require `WithFunctions(expr.CollectionFuncs())`: + +```go +// Sort numbers ascending, then reverse for descending. +reverse(sort([3, 1, 2])) // → [3, 2, 1] +sort(["banana", "apple"]) // → ["apple", "banana"] +``` + +`sort` accepts all-numbers or all-strings; mixed types produce +`ErrEvaluate`. It never mutates the input and returns a fresh `[]any`. +`reverse` works on any list type and also returns a fresh copy. diff --git a/docs/guides/higher-order-patterns.md b/docs/guides/higher-order-patterns.md index 43207bf..2cca571 100644 --- a/docs/guides/higher-order-patterns.md +++ b/docs/guides/higher-order-patterns.md @@ -1,40 +1,85 @@ # Higher-order patterns -`map`, `filter`, `any`, `all`, `find`, `count` are the closest thing -expr has to control flow over collections. There's no `for` and no -`let`. These six forms, plus the lazy `if(cond, t, f)` special form -(only the selected branch evaluates) and Go's short-circuit `&&` / -`||`, are how you make decisions and shape data. This guide walks the idioms that come up most often and the -ones you have to work around. +`map`, `filter`, `flatMap`, `any`, `all`, `find`, `count`, and `sortBy` +are the closest thing expr has to control flow over collections. There +is no `for` and no `let`. These eight forms, plus the lazy `if(cond, t, +f)` special form (only the selected branch evaluates) and Go's +short-circuit `&&` / `||`, are how you make decisions and shape data. +This guide walks the idioms that come up most often and the ones you +have to work around. A runnable companion lives in [`../../examples/higher_order_patterns/`](../../examples/higher_order_patterns/). ## The shape of a higher-order form +Every iterating form accepts two call shapes: + ``` -form(list, predicate_or_transform) +form(list, body) // two-arg: binds `it` and `index` +form(list, name, body) // three-arg: binds `name` and `index` ``` -- `list` must be a slice, array, or `nil`. **Maps are not iterated.** - To iterate a map, drive with `keys(m)` and index into `m[k]` inside - the predicate. -- The second argument is an **unevaluated AST** that the form - re-evaluates once per element. Inside that body, `it` is the - current element and `index` is its 0-based position. Both shadow - any outer identifier of the same name. +- `list` must be a slice, array, or `nil`. **Maps are not iterated + directly.** Use `keys(m)` to iterate keys, or `entries(m)` to iterate + key-value pairs. +- In the two-arg form, `it` is the current element and `index` is its + 0-based position. Both shadow any outer identifier of the same name. +- In the three-arg form, the second argument is the element binding name. + Only the chosen name and `index` are bound inside the body; `it` is + **not** bound, so an enclosing scope's `it` remains visible. + +These forms are **always registered**. `WithBuiltins()` is not required. +You can shadow any of them by registering your own function of the same +name, but you lose the per-element re-evaluation. + +## Named element bindings + +The three-arg form solves a problem the two-arg form cannot: nested +forms where you need to refer to the outer element by name from inside +an inner body. + +With the two-arg form only, the inner body shadows the outer `it`: + +``` +// Inside the inner map, `it` is a comment, not a review. +// There is no way to refer to the review from inside this body. +map(reviews, map(it.comments, it)) // it = comment here, review is gone +``` -These forms are **always registered**. `WithBuiltins()` is not -required. You can shadow them by registering your own function of -the same name, but you lose the per-element re-evaluation — see -[registering-functions.md](registering-functions.md). +With named bindings, you choose which name is visible where: + +``` +// Outer two-arg, inner named: outer `it` (the review) stays visible +// because the inner named form does not bind `it`. +map(reviews, map(it.comments, c, it.author + "/" + c)) +// ^^ outer `it` = review ^^ inner `c` = comment + +// Outer named, inner two-arg: `r` (the review) is visible inside +// the inner body alongside inner `it` (the comment). +map(reviews, r, join(map(r.comments, r.author + "/" + it), ",")) +// ^^ binds r ^^ inner `it` = comment +``` + +Named bindings shadow env names and outer bindings of the same name. +An inner form that reuses a name hides the outer one for its body: + +``` +map(users, u, map(u.orders, u, u)) // inner u shadows outer u +``` + +### Reserved binding names + +You cannot use `it`, `index`, `true`, `false`, `nil`, `map`, or `if` +as a binding name. Any of these produces an `ErrEvaluate: