diff --git a/README.md b/README.md index 40e90ef..0c86afd 100644 --- a/README.md +++ b/README.md @@ -130,6 +130,9 @@ where 1 = 1 order by id ``` +(A template may instead spell its binds the way sqlc does, giving up this property for +values in exchange for having them typed by an analyzer; see [Bind syntax](#bind-syntax).) + When the text is executed verbatim in a SQL client, `/*%if*/`, `/*%end*/`, and `/*name*/` are interpreted as comments, and the trailing literal `'Alice'` remains in place; the client therefore evaluates `name = 'Alice'`. When the same text is processed by bisql, the `/*%if*/` @@ -632,12 +635,105 @@ a `/*%for*/` directive is the iterable expression verbatim — including any col `a ? b : c`, a slice `x[1:2]`, a map `{k: v}`). The evaluator is replaceable through `WithEvaluator`. +## Bind syntax + +A bind is written as `/*expr*/literal` by default, and that shape is what makes a template +runnable as-is: the comment is ignored and the sample literal takes its place. The cost +appears when the template is also meant to be read by a static analyzer such as +[sqlc](https://sqlc.dev). Being runnable requires a **literal** at the bind site; being +recognized as a parameter requires a **marker**. No single text is both, so an analyzer +reading a two-way template sees constants where the binds are — it can check the SQL, the +catalog, and the result columns, but it can say nothing about the arguments. + +`WithBindSyntax(bindsyntax.SqlcNamed)` trades the runnable-as-is property, for values only, +to get that back: + +```go +tmpl, err := bisql.Parse(src, + bisql.WithBindSyntax(bindsyntax.SqlcNamed), + bisql.WithDialect(dialect.PostgreSQL)) +``` + +| Form | Binds | Notes | +| ---- | ----- | ----- | +| `sqlc.arg(name)` | one parameter | the name may also be quoted, and only then may it contain dots (`'c.name'`), for a value reached through a field | +| `sqlc.narg(name)` | one parameter | identical at build time; the distinction is for the analyzer | +| `sqlc.slice(name)` | a placeholder list | the parentheses stay in the template, as `in (sqlc.slice(ids))` | +| `@name` | one parameter | a shortcut for `sqlc.arg(name)`, **except under MySQL** (see below); `@a.b` is an error, not a dotted name | + +```sql +select id from users +where 1 = 1 + /*%if activeOnly*/ and status = @status /*%end*/ + /*%if minAge != null*/ and age >= @min_age /*%end*/ + /*%for kw in keywords*/ and name like @kw /*%end*/ + and id in (sqlc.slice('ids')) +``` + +**Only the bind spelling changes.** Every block directive is a SQL comment under either +syntax, so `/*%if*/`, `/*%elseif*/`, `/*%else*/`, `/*%for*/` and `@include` behave +identically, and placeholder numbering is the same single renderer-global counter — binds in +branches that did not render are still never counted. + +The two forms that depend on a test literal have no meaning without one, so `Parse` rejects +them rather than reinterpreting them: + +- **The two-way bind directive.** `/*status*/'active'` would otherwise be read as a plain + comment followed by a literal, producing a query that runs while ignoring a value. +- **`/*^ */` literal interpolation.** The value is inlined as text rather than bound, so an + analyzer sees a constant there and can check nothing about it. Bind the value as a + parameter, or use a whitelisted `/*%if*/` toggle for an identifier or a sort direction. + +A prefix that could only have been meant as a marker but cannot be one is rejected for the +same reason, since nothing downstream parses the SQL to catch it: + +```sql +where name = @c.name -- error: a dotted name must be sqlc.arg('c.name') +where name = sqlc.arg(c.name) -- error: a dotted name has to be quoted +``` + +`@c.name` would otherwise bind only `c` and render as `$1.name`. sqlc makes the same reading +and then rejects the edited query for being invalid SQL; bisql never parses the SQL, so it +has to reject the spelling up front instead. + +The two syntaxes are not mirror images of each other. A named marker is opaque under the +default syntax — `@status` is a plain word there — but a two-way directive under `SqlcNamed` +is an error rather than text, because reading it as a comment followed by a literal would +give a query that runs while ignoring a value. + +### The `@name` shortcut and MySQL + +**`@name` is a bind under every dialect except MySQL**, where it stays ordinary text. This +mirrors sqlc, which does not support the shortcut for MySQL because `@name` there is a user +variable — and mirroring it is the point: a spelling one of them binds and the other does +not is precisely the divergence this arrangement exists to avoid. + +```sql +-- sqlc-named, MySQL: a user variable, left alone +select @row_number := @row_number + 1 +-- sqlc-named, PostgreSQL: two binds +select @row_number := @row_number + 1 -- renders as: select $1 := $2 + 1 +``` + +Recognizing `@name` also requires that what follows the `@` can start an identifier, so `@>` +and `@@version` are names under no dialect. The call forms are available everywhere, so a +template that has to read the same way under both engines should use them. + +### What is not a bind + +`sqlc.embed(table)` is a result-column construct — it selects a whole table into a nested +struct — so it is not a bind and passes through untouched. Note that sqlc expands it into an +explicit column list in the SQL it hands back, so a tool rendering that SQL never sees the +call; a tool rendering the *original* template would, and would send it to the database. + ## Package layout ```text bisql Public API: NewParser, Parser, Parse, ParseFile, Expand, ExpandFile, Template, Statement, Option, and fragment loaders (Loader, RegistryLoader, FSLoader, LoaderFunc, StackedLoader, ErrNotFound, WithLoader, WithStackedLoader). +bindsyntax/ How a bind is written: TwoWay (the default /*expr*/literal form) and + SqlcNamed (@name / sqlc.arg('name'), not implemented yet). dialect/ Dialect definitions: placeholder generation and literal formatting (MySQL, SQLite, PostgreSQL, Oracle, SQL Server). expr/ Evaluator interface and Scope (for custom evaluators). diff --git a/bindsyntax/bindsyntax.go b/bindsyntax/bindsyntax.go new file mode 100644 index 0000000..c178825 --- /dev/null +++ b/bindsyntax/bindsyntax.go @@ -0,0 +1,290 @@ +// Package bindsyntax selects how a bind is written in a template. +// +// bisql's own syntax puts the bind in a comment and leaves a sample literal in +// the SQL — `/*status*/'active'` — which is what makes a template runnable as-is +// in a client: the comment is ignored and the literal takes its place. That +// property is why the syntax looks the way it does. +// +// The cost of it shows up when a template is also meant to be read by a static +// analyzer such as sqlc. Being runnable requires a *literal* at the bind site; +// being recognized as a parameter requires a *marker*. No single text is both, so +// a template written the two-way way is one whose binds an analyzer sees as +// constants — it can check the SQL and the result columns, but it can tell you +// nothing about the arguments. +// +// SqlcNamed gives up the runnable-as-is property for values, and only for +// values, in exchange for that. Structure directives are unaffected either way: +// `/*%if*/` and `/*%for*/` are comments under any bind syntax, so an analyzer +// skips them without needing to understand them. What it buys is that every bind +// becomes a parameter the analyzer resolves against the catalog, with a name +// attached. +// +// two-way where status = /*status*/'active' +// sqlc-named where status = @status +// where name like sqlc.arg('c.name') -- a dotted name, for a loop element +// +// Recognize is the seam a lexer calls: it decides whether a bind marker starts +// here, and says nothing about the surrounding SQL. +package bindsyntax + +import ( + "fmt" + "strings" +) + +// Syntax is a choice of bind spelling. +type Syntax uint8 + +const ( + // TwoWay is bisql's own syntax, `/*expr*/literal`, and the default. A template + // written this way runs unmodified in a SQL client. + TwoWay Syntax = iota + + // SqlcNamed is sqlc's syntax: `@name`, `sqlc.arg('name')`, `sqlc.narg('name')`, + // `sqlc.slice('name')`. A template written this way is not runnable as-is, but + // its binds are parameters a static analyzer can resolve and type. + SqlcNamed +) + +func (s Syntax) String() string { + switch s { + case TwoWay: + return "two-way" + case SqlcNamed: + return "sqlc-named" + } + return "unknown" +} + +// Rules are the bind spellings a template may use, resolved from the syntax and the +// dialect. The resolution exists because one spelling is not available everywhere: sqlc +// supports @name as a shortcut for sqlc.arg(name) for every engine except MySQL, where +// @name is a user variable. bisql has to read a template exactly the way sqlc does — a +// spelling one of them binds and the other does not is the one failure this design is +// built to avoid — so the shortcut has to be dialect-dependent here too. +type Rules struct { + Syntax Syntax + AtForm bool // whether a bare @name is a bind +} + +// RulesFor resolves the rules for a syntax and a dialect name (as dialect.Dialect.Name +// reports it). +func RulesFor(s Syntax, dialectName string) Rules { + return Rules{Syntax: s, AtForm: s == SqlcNamed && dialectName != "mysql"} +} + +// Kind distinguishes the SqlcNamed forms, which differ in what they promise about +// the value rather than in how it is spelled. +type Kind uint8 + +const ( + // Arg is a plain parameter: `@name` or `sqlc.arg('name')`. + Arg Kind = iota + // NArg is a parameter that may be null: `sqlc.narg('name')`. + NArg + // Slice is a parameter that expands to a placeholder list: `sqlc.slice('name')`. + Slice +) + +func (k Kind) String() string { + switch k { + case Arg: + return "arg" + case NArg: + return "narg" + case Slice: + return "slice" + } + return "unknown" +} + +// Marker is a recognized bind: the name it binds, which form it took, and how +// many bytes of the input it spans. +type Marker struct { + Name string + Kind Kind + Len int +} + +// callForms are the wrapper calls, longest-distinguishing prefix first so that narg is not +// read as a suffix of anything. +var callForms = []struct { + prefix string + kind Kind +}{ + {"sqlc.arg(", Arg}, + {"sqlc.narg(", NArg}, + {"sqlc.slice(", Slice}, +} + +// Recognize reports the bind marker at the start of s, if there is one. It recognizes +// exactly what sqlc recognizes, including that a bare @name ends at the first character +// that cannot continue an identifier — so @a.b yields the marker @a, as it does for sqlc. +// Malformed is what tells a caller that such a prefix was a mistake rather than a bind. +// +// Recognize looks only at s's prefix and never scans ahead, so a caller that already tracks +// quotes and comments — as a lexer does — can consult it at each position without giving up +// that tracking. +func (r Rules) Recognize(s string) (Marker, bool) { + if r.Syntax != SqlcNamed { + return Marker{}, false + } + if r.AtForm { + if name, n, ok := atName(s); ok { + return Marker{Name: name, Kind: Arg, Len: n}, true + } + } + for _, form := range callForms { + if !strings.HasPrefix(s, form.prefix) { + continue + } + name, n, ok := callArgument(s[len(form.prefix):]) + if !ok { + return Marker{}, false + } + return Marker{Name: name, Kind: form.kind, Len: len(form.prefix) + n}, true + } + return Marker{}, false +} + +// Malformed reports a reason when s begins with something that can only have been meant as +// a bind marker but cannot be one, so that a caller can fail on the mistake instead of +// passing it through. +// +// Both cases it catches would otherwise become SQL that looks plausible and is not. A +// dotted @a.b binds only "a" and leaves ".b" behind, which renders as "$1.b"; sqlc makes +// the same reading and then rejects the result, but a renderer that never parses SQL has +// nothing to reject it with. A call form whose argument is not a single-quoted name matches +// nothing and is emitted verbatim, becoming a call to a function that does not exist. +// +// It should be consulted before Recognize, since Recognize accepts the leading @a of a +// dotted name. +func (r Rules) Malformed(s string) (string, bool) { + if r.Syntax != SqlcNamed { + return "", false + } + if name, n, ok := atName(s); ok && r.AtForm { + if n < len(s) && s[n] == '.' { + return fmt.Sprintf("@%s is followed by a period: a dotted bind name has to be "+ + "written as sqlc.arg('%s.…'), because @%s.… reads as the parameter @%s and "+ + "then trailing text", name, name, name, name), true + } + return "", false + } + for _, form := range callForms { + if !strings.HasPrefix(s, form.prefix) { + continue + } + rest := s[len(form.prefix):] + if _, _, ok := callArgument(rest); ok { + return "", false + } + // An unquoted dotted name is the likeliest mistake, and it has a better fix than + // the general one: quote it. + if name, n, ok := leadingIdent(rest); ok && n < len(rest) && rest[n] == '.' { + return fmt.Sprintf("%s%s.…) has a dotted name, which has to be quoted: %s'%s.…')", + form.prefix, name, form.prefix, name), true + } + return fmt.Sprintf("%s…) takes a name, either bare as %sname) or quoted as %s'a.name')", + form.prefix, form.prefix, form.prefix), true + } + return "", false +} + +// atName reads an `@name` marker. The name is a bare identifier: a dotted name +// has to be written as sqlc.arg('a.b'), because `@a.b` is not valid input to sqlc +// either. +func atName(s string) (string, int, bool) { + if !strings.HasPrefix(s, "@") { + return "", 0, false + } + i := 1 + for i < len(s) && isIdent(s[i], i == 1) { + i++ + } + if i == 1 { + return "", 0, false + } + return s[1:i], i, true +} + +// callArgument reads the argument of a wrapper call, `name)` or `'name')`, allowing space +// around it. sqlc accepts both spellings but only the quoted one may carry dots — an +// unquoted a.b parses as a column reference and sqlc rejects it — so the same holds here. +func callArgument(s string) (string, int, bool) { + if name, n, ok := bareName(s); ok { + return name, n, true + } + return quotedName(s) +} + +// bareName reads `name)`: an unquoted identifier, with no dots. +func bareName(s string) (string, int, bool) { + name, i, ok := leadingIdent(s) + if !ok { + return "", 0, false + } + i = skipSpace(s, i) + if i >= len(s) || s[i] != ')' { + return "", 0, false + } + return name, i + 1, true +} + +// leadingIdent reads the identifier at the start of s, after any space, and reports how far +// it read. It does not care what follows, which is what lets a caller tell `name)` from +// `name.other)`. +func leadingIdent(s string) (string, int, bool) { + i := skipSpace(s, 0) + start := i + for i < len(s) && isIdent(s[i], i == start) { + i++ + } + if i == start { + return "", 0, false + } + return s[start:i], i, true +} + +// quotedName reads `'name')`, allowing space around the literal. The name is +// taken verbatim up to the closing quote, so it may contain dots. +func quotedName(s string) (string, int, bool) { + i := skipSpace(s, 0) + if i >= len(s) || s[i] != '\'' { + return "", 0, false + } + i++ + start := i + for i < len(s) && s[i] != '\'' { + i++ + } + if i >= len(s) || i == start { + return "", 0, false + } + name := s[start:i] + i++ // closing quote + i = skipSpace(s, i) + if i >= len(s) || s[i] != ')' { + return "", 0, false + } + return name, i + 1, true +} + +func skipSpace(s string, i int) int { + for i < len(s) && (s[i] == ' ' || s[i] == '\t' || s[i] == '\n' || s[i] == '\r') { + i++ + } + return i +} + +// isIdent reports whether c may appear in an identifier, at the first position or +// after it. +func isIdent(c byte, first bool) bool { + switch { + case c >= 'a' && c <= 'z', c >= 'A' && c <= 'Z', c == '_': + return true + case c >= '0' && c <= '9': + return !first + } + return false +} diff --git a/bindsyntax/bindsyntax_test.go b/bindsyntax/bindsyntax_test.go new file mode 100644 index 0000000..2f1765b --- /dev/null +++ b/bindsyntax/bindsyntax_test.go @@ -0,0 +1,171 @@ +package bindsyntax_test + +import ( + "strings" + "testing" + + "github.com/mpyw/bisql/bindsyntax" +) + +// pg and my are the rules a PostgreSQL and a MySQL template are read under: the @name +// shortcut exists for one and not the other, because that is how sqlc reads them. +var ( + pg = bindsyntax.RulesFor(bindsyntax.SqlcNamed, "postgresql") + my = bindsyntax.RulesFor(bindsyntax.SqlcNamed, "mysql") +) + +func TestRecognize(t *testing.T) { + cases := []struct { + in string + name string + kind bindsyntax.Kind + n int + }{ + {"@status", "status", bindsyntax.Arg, 7}, + {"@status and x = 1", "status", bindsyntax.Arg, 7}, + {"@_leading", "_leading", bindsyntax.Arg, 9}, + {"@a1", "a1", bindsyntax.Arg, 3}, + {"sqlc.arg(status)", "status", bindsyntax.Arg, 16}, + {"sqlc.narg(note)", "note", bindsyntax.NArg, 15}, + {"sqlc.slice(ids)", "ids", bindsyntax.Slice, 15}, + {"sqlc.arg('status')", "status", bindsyntax.Arg, 18}, + {"sqlc.narg('note')", "note", bindsyntax.NArg, 17}, + {"sqlc.slice('ids')", "ids", bindsyntax.Slice, 17}, + {"sqlc.arg('c.name')", "c.name", bindsyntax.Arg, 18}, + {"sqlc.arg( 'x' )", "x", bindsyntax.Arg, 15}, + // A cast follows the marker and is not part of it. + {"sqlc.arg('x')::text", "x", bindsyntax.Arg, 13}, + } + for _, c := range cases { + t.Run(c.in, func(t *testing.T) { + m, ok := pg.Recognize(c.in) + if !ok { + t.Fatalf("Recognize(%q) = not recognized", c.in) + } + if m.Name != c.name || m.Kind != c.kind || m.Len != c.n { + t.Errorf("Recognize(%q) = %+v, want {Name:%q Kind:%v Len:%d}", c.in, m, c.name, c.kind, c.n) + } + if got := c.in[:m.Len]; len(got) != c.n { + t.Errorf("Len %d does not span a prefix of %q", m.Len, c.in) + } + }) + } +} + +func TestRecognizeRejects(t *testing.T) { + // @a.b is rejected because sqlc rejects it too: a dotted name has to be written + // as sqlc.arg('a.b'). Recognizing @a here would silently bind something else. + for _, in := range []string{ + "", "@", "@1abc", "@ ", "status", "'@status'", + "sqlc.arg()", "sqlc.arg('')", "sqlc.arg('x'", "sqlc.args('x')", + "sqlc.arg(\"x\")", "sqlc.slice('x'", "@@x", + } { + t.Run(in, func(t *testing.T) { + if m, ok := pg.Recognize(in); ok { + t.Errorf("Recognize(%q) = %+v, want not recognized", in, m) + } + }) + } +} + +// @a.b stops at the dot, which is the reading sqlc makes too — and why Recognize alone is +// not enough to tell a bind from a mistake. +func TestRecognizeStopsAtDot(t *testing.T) { + m, ok := pg.Recognize("@a.b") + if !ok || m.Name != "a" || m.Len != 2 { + t.Errorf("Recognize(\"@a.b\") = %+v, %v; want name a spanning 2 bytes", m, ok) + } +} + +// Malformed is what makes the two silent mistakes loud. Without it @c.name renders as +// "$1.name" and sqlc.arg(x) is emitted verbatim as a call to a function that does not +// exist, both without any complaint. +func TestMalformed(t *testing.T) { + for _, c := range []struct{ in, want string }{ + {"@c.name", "dotted bind name"}, + {"@c.name = 1", "dotted bind name"}, + {"sqlc.arg(c.name)", "has to be quoted"}, + {`sqlc.arg("x")`, "takes a name"}, + {"sqlc.arg()", "takes a name"}, + {"sqlc.arg('')", "takes a name"}, + {"sqlc.arg('x'", "takes a name"}, + {"sqlc.narg(c.note)", "has to be quoted"}, + } { + t.Run(c.in, func(t *testing.T) { + reason, bad := pg.Malformed(c.in) + if !bad { + t.Fatalf("Malformed(%q) = not malformed", c.in) + } + if !strings.Contains(reason, c.want) { + t.Errorf("reason = %q, want it to contain %q", reason, c.want) + } + }) + } +} + +// A well-formed marker is not malformed, and neither is anything that was never a marker: +// @ and sqlc. both occur in ordinary SQL. +func TestMalformedLeavesEverythingElseAlone(t *testing.T) { + for _, in := range []string{ + "@status", "@status = 1", "sqlc.arg('x')", "sqlc.narg('c.note')", "sqlc.slice('ids')", + "sqlc.arg(x)", "sqlc.narg(note)", "sqlc.slice(ids)", + "", "@", "@>", "@@version", "tags @> '{a}'", "status", "sqlc.args('x')", "sqlc_arg('x')", + // sqlc.embed is a result-column construct, not a bind: it stays opaque. + "sqlc.embed(authors)", + } { + t.Run(in, func(t *testing.T) { + if reason, bad := pg.Malformed(in); bad { + t.Errorf("Malformed(%q) = %q, want not malformed", in, reason) + } + }) + } +} + +func TestStrings(t *testing.T) { + if got := bindsyntax.TwoWay.String(); got != "two-way" { + t.Errorf("TwoWay = %q", got) + } + if got := bindsyntax.SqlcNamed.String(); got != "sqlc-named" { + t.Errorf("SqlcNamed = %q", got) + } + if got := bindsyntax.NArg.String(); got != "narg" { + t.Errorf("NArg = %q", got) + } +} + +// sqlc does not support the @name shortcut for MySQL, where @name is a user variable, so +// neither does bisql: a spelling one of them binds and the other does not is the divergence +// this whole arrangement exists to avoid. +func TestRulesForMySQLHasNoAtForm(t *testing.T) { + if my.AtForm { + t.Fatal("MySQL rules must not enable the @name form") + } + if !pg.AtForm { + t.Fatal("PostgreSQL rules must enable the @name form") + } + for _, in := range []string{"@status", "@row_number"} { + if m, ok := my.Recognize(in); ok { + t.Errorf("my.Recognize(%q) = %+v, want not recognized", in, m) + } + if reason, bad := my.Malformed(in); bad { + t.Errorf("my.Malformed(%q) = %q, want not malformed", in, reason) + } + } + // The call forms are available for every dialect. + if m, ok := my.Recognize("sqlc.arg(status)"); !ok || m.Name != "status" { + t.Errorf("my.Recognize(call form) = %+v, %v", m, ok) + } +} + +// The two-way syntax recognizes nothing here at all. +func TestRulesForTwoWayRecognizesNothing(t *testing.T) { + r := bindsyntax.RulesFor(bindsyntax.TwoWay, "postgresql") + for _, in := range []string{"@status", "sqlc.arg('x')", "sqlc.arg(c.name)"} { + if m, ok := r.Recognize(in); ok { + t.Errorf("Recognize(%q) = %+v, want not recognized", in, m) + } + if reason, bad := r.Malformed(in); bad { + t.Errorf("Malformed(%q) = %q, want not malformed", in, reason) + } + } +} diff --git a/bindsyntax_option_test.go b/bindsyntax_option_test.go new file mode 100644 index 0000000..000c275 --- /dev/null +++ b/bindsyntax_option_test.go @@ -0,0 +1,312 @@ +package bisql_test + +import ( + "reflect" + "strings" + "testing" + + "github.com/mpyw/bisql" + "github.com/mpyw/bisql/bindsyntax" + "github.com/mpyw/bisql/dialect" +) + +func TestWithBindSyntax_defaultIsTwoWay(t *testing.T) { + tmpl, err := bisql.Parse("select 1 where x = /*x*/'a'") + if err != nil { + t.Fatalf("parse: %v", err) + } + stmt, err := tmpl.Build(map[string]any{"x": "b"}) + if err != nil { + t.Fatalf("build: %v", err) + } + if stmt.SQL != "select 1 where x = ?" || len(stmt.Args) != 1 { + t.Errorf("SQL = %q, Args = %v", stmt.SQL, stmt.Args) + } +} + +func TestSqlcNamed(t *testing.T) { + cases := []struct { + name string + src string + params map[string]any + sql string + args []any + }{ + { + name: "at form", + src: "select id from users where status = @status", + params: map[string]any{"status": "active"}, + sql: "select id from users where status = $1", + args: []any{"active"}, + }, + { + name: "call form, bare name", + src: "select id from users where status = sqlc.arg(status)", + params: map[string]any{"status": "active"}, + sql: "select id from users where status = $1", + args: []any{"active"}, + }, + { + name: "call form, quoted name", + src: "select id from users where status = sqlc.arg('status')", + params: map[string]any{"status": "active"}, + sql: "select id from users where status = $1", + args: []any{"active"}, + }, + { + name: "nullable form binds like any other", + src: "update users set note = sqlc.narg('note') where id = @id", + params: map[string]any{"note": nil, "id": 7}, + sql: "update users set note = $1 where id = $2", + args: []any{nil, 7}, + }, + { + // A dotted name is why the call form exists; @c.name is rejected outright. + name: "dotted name reaches into a value", + src: "select id from users where name = sqlc.arg('c.name')", + params: map[string]any{"c": map[string]any{"name": "ada"}}, + sql: "select id from users where name = $1", + args: []any{"ada"}, + }, + { + // Without a test literal to read the shape from, the slice form is what asks + // for expansion; a plain arg binds the slice whole, as an array parameter. + name: "slice form expands into a list", + src: "select id from users where id in (sqlc.slice('ids'))", + params: map[string]any{"ids": []any{1, 2, 3}}, + sql: "select id from users where id in ($1, $2, $3)", + args: []any{1, 2, 3}, + }, + { + name: "arg form binds a slice as one parameter", + src: "select id from users where id = any(@ids)", + params: map[string]any{"ids": []int{1, 2}}, + sql: "select id from users where id = any($1)", + args: []any{[]int{1, 2}}, + }, + { + name: "a trailing cast is ordinary text", + src: "select id from users where name = sqlc.arg('name')::text", + params: map[string]any{"name": "ada"}, + sql: "select id from users where name = $1::text", + args: []any{"ada"}, + }, + { + // The block directives are comments under either syntax, so numbering stays + // gap-free across branches that did not render. + name: "directives behave identically", + src: "select id from users\nwhere 1 = 1\n" + + " /*%if activeOnly*/ and status = @status /*%end*/\n" + + " /*%if minAge != null*/ and age >= @min_age /*%end*/\n" + + " /*%for kw in keywords*/ and name like @kw /*%end*/", + params: map[string]any{ + "activeOnly": false, "minAge": 20, "min_age": 20, + "keywords": []any{"%a%", "%b%"}, + }, + sql: "select id from users\nwhere 1 = 1\n" + + " \n" + + " and age >= $1 \n" + + " and name like $2 and name like $3 ", + args: []any{20, "%a%", "%b%"}, + }, + { + // @> is an operator, not a bind: recognition stops when what follows @ cannot + // start an identifier. + name: "an operator that starts with @ is left alone", + src: "select id from users where tags @> @tags", + params: map[string]any{"tags": "{a}"}, + sql: "select id from users where tags @> $1", + args: []any{"{a}"}, + }, + { + name: "a marker inside a string literal is text", + src: "select '@status' as lit where x = @x", + params: map[string]any{"x": 1}, + sql: "select '@status' as lit where x = $1", + args: []any{1}, + }, + } + + for _, c := range cases { + t.Run(c.name, func(t *testing.T) { + tmpl, err := bisql.Parse(c.src, + bisql.WithBindSyntax(bindsyntax.SqlcNamed), + bisql.WithDialect(dialect.PostgreSQL)) + if err != nil { + t.Fatalf("parse: %v", err) + } + stmt, err := tmpl.Build(c.params) + if err != nil { + t.Fatalf("build: %v", err) + } + if stmt.SQL != c.sql { + t.Errorf("SQL\n got: %q\nwant: %q", stmt.SQL, c.sql) + } + if !reflect.DeepEqual(stmt.Args, c.args) { + t.Errorf("Args\n got: %#v\nwant: %#v", stmt.Args, c.args) + } + }) + } +} + +// The two forms that read a test literal have no meaning without one, and silently +// reinterpreting them would produce a query that runs while ignoring a value. +func TestSqlcNamed_rejectsTheTwoWayForms(t *testing.T) { + cases := []struct { + name string + src string + want string + }{ + { + name: "two-way bind directive", + src: "select id from users where status = /*status*/'active'", + want: "the two-way bind directive", + }, + { + name: "literal interpolation", + src: "select id from users limit /*^lim*/10", + want: "literal interpolation", + }, + } + for _, c := range cases { + t.Run(c.name, func(t *testing.T) { + _, err := bisql.Parse(c.src, bisql.WithBindSyntax(bindsyntax.SqlcNamed)) + if err == nil { + t.Fatalf("want an error containing %q, got nil", c.want) + } + if !strings.Contains(err.Error(), c.want) { + t.Errorf("error = %q, want it to contain %q", err, c.want) + } + if !strings.Contains(err.Error(), "sqlc-named") { + t.Errorf("error = %q, want it to name the syntax", err) + } + }) + } +} + +// A prefix that could only have been meant as a marker but cannot be one is a mistake, and +// this is the only place it can be caught: nothing downstream parses the SQL, so @c.name +// would otherwise render as "$1.name" and sqlc.arg(x) would be emitted verbatim. +func TestSqlcNamed_rejectsMalformedMarkers(t *testing.T) { + cases := []struct { + name string + src string + want string + }{ + { + name: "dotted at-name", + src: "select id from users where name = @c.name", + want: "dotted bind name", + }, + { + name: "unquoted dotted call argument", + src: "select id from users where name = sqlc.arg(c.name)", + want: "has to be quoted", + }, + { + name: "double-quoted call argument", + src: `select id from users where name = sqlc.arg("x")`, + want: "takes a name", + }, + { + name: "empty call argument", + src: "select id from users where name = sqlc.arg()", + want: "takes a name", + }, + } + for _, c := range cases { + t.Run(c.name, func(t *testing.T) { + _, err := bisql.Parse(c.src, + bisql.WithBindSyntax(bindsyntax.SqlcNamed), + bisql.WithDialect(dialect.PostgreSQL)) + if err == nil { + t.Fatalf("want an error containing %q, got nil", c.want) + } + if !strings.Contains(err.Error(), c.want) { + t.Errorf("error = %q, want it to contain %q", err, c.want) + } + }) + } +} + +// A named marker is not a bind under the two-way syntax; it stays opaque text. Note that +// the reverse does not hold: a two-way directive under SqlcNamed is an error, not text. +func TestTwoWay_leavesNamedMarkersAlone(t *testing.T) { + tmpl, err := bisql.Parse("select @status, tags @> '{a}'") + if err != nil { + t.Fatalf("parse: %v", err) + } + stmt, err := tmpl.Build(nil) + if err != nil { + t.Fatalf("build: %v", err) + } + if stmt.SQL != "select @status, tags @> '{a}'" || len(stmt.Args) != 0 { + t.Errorf("SQL = %q, Args = %v", stmt.SQL, stmt.Args) + } +} + +// sqlc does not support the @name shortcut for MySQL, where @name is a user variable, so +// bisql must not read one as a bind there either: a spelling one of them binds and the +// other does not is the divergence this arrangement exists to avoid. The same text under +// PostgreSQL, where sqlc does support the shortcut, is a bind. +func TestSqlcNamed_atFormFollowsTheDialect(t *testing.T) { + const src = "select @row_number := @row_number + 1" + + tmpl, err := bisql.Parse(src, + bisql.WithBindSyntax(bindsyntax.SqlcNamed), bisql.WithDialect(dialect.MySQL)) + if err != nil { + t.Fatalf("parse (mysql): %v", err) + } + stmt, err := tmpl.Build(nil) + if err != nil { + t.Fatalf("build (mysql): %v", err) + } + if stmt.SQL != src || len(stmt.Args) != 0 { + t.Errorf("mysql: SQL = %q, Args = %v; want the user variable left alone", stmt.SQL, stmt.Args) + } + + tmpl, err = bisql.Parse(src, + bisql.WithBindSyntax(bindsyntax.SqlcNamed), bisql.WithDialect(dialect.PostgreSQL)) + if err != nil { + t.Fatalf("parse (postgres): %v", err) + } + stmt, err = tmpl.Build(map[string]any{"row_number": 1}) + if err != nil { + t.Fatalf("build (postgres): %v", err) + } + if stmt.SQL != "select $1 := $2 + 1" { + t.Errorf("postgres: SQL = %q, want the @name read as a bind", stmt.SQL) + } + + // A double-@ session variable and the @> operator are names under no dialect. + tmpl, err = bisql.Parse("select @@version, tags @> '{a}'", + bisql.WithBindSyntax(bindsyntax.SqlcNamed), bisql.WithDialect(dialect.PostgreSQL)) + if err != nil { + t.Fatalf("parse: %v", err) + } + stmt, err = tmpl.Build(nil) + if err != nil { + t.Fatalf("build: %v", err) + } + if stmt.SQL != "select @@version, tags @> '{a}'" || len(stmt.Args) != 0 { + t.Errorf("SQL = %q, Args = %v", stmt.SQL, stmt.Args) + } +} + +// sqlc.embed selects a whole table into a nested struct; it is a result-column construct, +// not a bind, so it has to pass through untouched. +func TestSqlcNamed_leavesEmbedAlone(t *testing.T) { + const src = "select sqlc.embed(authors), id from authors where id = @id" + tmpl, err := bisql.Parse(src, + bisql.WithBindSyntax(bindsyntax.SqlcNamed), bisql.WithDialect(dialect.PostgreSQL)) + if err != nil { + t.Fatalf("parse: %v", err) + } + stmt, err := tmpl.Build(map[string]any{"id": 1}) + if err != nil { + t.Fatalf("build: %v", err) + } + if stmt.SQL != "select sqlc.embed(authors), id from authors where id = $1" { + t.Errorf("SQL = %q, want sqlc.embed untouched", stmt.SQL) + } +} diff --git a/bisql.go b/bisql.go index 026fff9..cd9a74d 100644 --- a/bisql.go +++ b/bisql.go @@ -6,6 +6,7 @@ import ( "reflect" "strings" + "github.com/mpyw/bisql/bindsyntax" "github.com/mpyw/bisql/dialect" "github.com/mpyw/bisql/expr" "github.com/mpyw/bisql/internal/exprlang" @@ -64,9 +65,10 @@ func (s Statement) SQLWithArgs() string { type Option func(*config) type config struct { - dialect dialect.Dialect - evaluator expr.Evaluator - loader Loader + dialect dialect.Dialect + evaluator expr.Evaluator + loader Loader + bindSyntax bindsyntax.Syntax } func defaultConfig() config { @@ -76,6 +78,21 @@ func defaultConfig() config { // WithDialect sets the dialect used for placeholder generation (default: MySQL). func WithDialect(d dialect.Dialect) Option { return func(c *config) { c.dialect = d } } +// WithBindSyntax selects how binds are written in the template (default: +// bindsyntax.TwoWay, bisql's own /*expr*/literal form). +// +// Only the bind spelling changes: the block directives are SQL comments under either +// syntax, so /*%if*/ and /*%for*/ behave identically. Under bindsyntax.SqlcNamed the two +// forms that depend on a test literal are rejected rather than reinterpreted — the two-way +// bind directive itself, and /*^ */ literal interpolation. +// +// Which spellings are available also depends on the dialect, because sqlc's do: the @name +// shortcut is not supported for MySQL, where @name is a user variable. See the bindsyntax +// package. +func WithBindSyntax(s bindsyntax.Syntax) Option { + return func(c *config) { c.bindSyntax = s } +} + // WithEvaluator swaps the expression evaluator (default: the built-in one). func WithEvaluator(e expr.Evaluator) Option { return func(c *config) { c.evaluator = e } } @@ -124,7 +141,8 @@ func (p *Parser) Parse(src string) (*Template, error) { if err != nil { return nil, err } - root, err := parser.Parse(expanded) + root, err := parser.ParseWithRules(expanded, + bindsyntax.RulesFor(p.c.bindSyntax, p.c.dialect.Name())) if err != nil { return nil, err } diff --git a/internal/sqltmpl/ast/ast.go b/internal/sqltmpl/ast/ast.go index b1ff3c0..5b1bf09 100644 --- a/internal/sqltmpl/ast/ast.go +++ b/internal/sqltmpl/ast/ast.go @@ -104,19 +104,34 @@ type ForDirective struct { func (n ForDirective) Text() string { return n.Token + join(n.Nodes) } -// BindValue is /* expr */literal. Test is the test literal (a Word or Paren) that follows; -// it keeps the raw template runnable and is replaced at build time by a placeholder — or, -// when Test is a Paren, by an expanded IN list. Trailing is any content that folded in -// after the test literal (e.g. a "::cast"). +// BindValue is a bind. Under the two-way syntax it is /* expr */literal, where Test is the +// test literal (a Word or Paren) that follows: it keeps the raw template runnable and is +// replaced at build time by a placeholder — or, when Test is a Paren, by an expanded IN +// list. Trailing is any content that folded in after the test literal (e.g. a "::cast"). +// +// A syntax that spells the bind in the SQL instead carries its own name and has no test +// literal to infer anything from, so Test is nil there and ExpandList says outright whether +// the bind expands into a placeholder list. type BindValue struct { Loc Location Token string Expression string - Test Node + Test Node // nil when the syntax carries no test literal Trailing []Node + // ExpandList expands the value into a comma-separated run of placeholders. Unlike a + // parenthesized test literal, which the renderer replaces parens and all, this emits + // no parentheses: the template already carries them, because it had to be valid SQL + // before rendering. + ExpandList bool } -func (n BindValue) Text() string { return n.Token + n.Test.Text() + join(n.Trailing) } +func (n BindValue) Text() string { + s := n.Token + if n.Test != nil { + s += n.Test.Text() + } + return s + join(n.Trailing) +} // LiteralValue is /*^ expr */literal. type LiteralValue struct { diff --git a/internal/sqltmpl/lexer/lexer.go b/internal/sqltmpl/lexer/lexer.go index 49e76da..e8f346d 100644 --- a/internal/sqltmpl/lexer/lexer.go +++ b/internal/sqltmpl/lexer/lexer.go @@ -10,15 +10,17 @@ import ( "fmt" "unicode/utf8" + "github.com/mpyw/bisql/bindsyntax" "github.com/mpyw/bisql/internal/sqltmpl/ast" "github.com/mpyw/bisql/internal/sqltmpl/token" ) // Lexer scans a SQL template and yields tokens one at a time. type Lexer struct { - src string - pos int // scan position (byte offset) - line int // current line (1-based), tracked as newlines are consumed + src string + rules bindsyntax.Rules + pos int // scan position (byte offset) + line int // current line (1-based), tracked as newlines are consumed lineStart int // byte offset of the current line start tokenLine int // line at the start of the current token @@ -28,9 +30,14 @@ type Lexer struct { err error } -// New creates a Lexer over src. +// New creates a Lexer over src using bisql's two-way bind syntax. func New(src string) *Lexer { - return &Lexer{src: src, line: 1, lineStart: 0} + return NewWithRules(src, bindsyntax.Rules{Syntax: bindsyntax.TwoWay}) +} + +// NewWithRules creates a Lexer over src that recognizes the bind spellings the rules allow. +func NewWithRules(src string, rules bindsyntax.Rules) *Lexer { + return &Lexer{src: src, rules: rules, line: 1, lineStart: 0} } // Token returns the string of the most recently read token. @@ -128,6 +135,21 @@ func (l *Lexer) scan() token.Kind { return l.scanSlashStar() } + // A bind spelled in the SQL rather than in a comment is opaque text, so it has to be + // recognized before the surrounding word absorbs it. A prefix that can only have been + // meant as a marker but cannot be one is a mistake, and failing here is the only place + // it can be caught: bisql never parses the SQL, so nothing downstream would notice. + { + rest := l.src[l.pos:] + if reason, bad := l.rules.Malformed(rest); bad { + return l.fail("%s", reason) + } + if m, ok := l.rules.Recognize(rest); ok { + l.advanceOver(m.Len) + return token.NamedBind + } + } + // A word (identifier / keyword-like / number), possibly absorbing a '...' literal. if isWordStart(l.src, l.pos) { return l.scanWord() @@ -139,6 +161,18 @@ func (l *Lexer) scan() token.Kind { return token.Other } +// advanceOver consumes n bytes, keeping the line and column bookkeeping honest: a bind +// marker may be written across lines (sqlc.arg(\n'x')). +func (l *Lexer) advanceOver(n int) { + for i := 0; i < n; i++ { + if l.src[l.pos+i] == '\n' { + l.line++ + l.lineStart = l.pos + i + 1 + } + } + l.pos += n +} + func (l *Lexer) peekAt(i int) byte { if i < 0 || i >= len(l.src) { return 0 diff --git a/internal/sqltmpl/lexer/named_test.go b/internal/sqltmpl/lexer/named_test.go new file mode 100644 index 0000000..f76e35c --- /dev/null +++ b/internal/sqltmpl/lexer/named_test.go @@ -0,0 +1,110 @@ +package lexer_test + +import ( + "testing" + + "github.com/mpyw/bisql/bindsyntax" + "github.com/mpyw/bisql/internal/sqltmpl/lexer" + "github.com/mpyw/bisql/internal/sqltmpl/token" +) + +// scanAllNamed drains a lexer reading binds in sqlc's syntax. +func scanAllNamed(t *testing.T, src string) []tk { + t.Helper() + l := lexer.NewWithRules(src, bindsyntax.RulesFor(bindsyntax.SqlcNamed, "postgresql")) + var out []tk + for { + k := l.Next() + switch k { + case token.EOF: + return out + case token.Illegal: + return append(out, tk{token.Illegal, ""}) + } + out = append(out, tk{k, l.Token()}) + } +} + +// named returns the text of every bind marker the lexer recognized. +func named(toks []tk) []string { + var out []string + for _, tok := range toks { + if tok.kind == token.NamedBind { + out = append(out, tok.text) + } + } + return out +} + +func TestNamedBind(t *testing.T) { + cases := []struct { + src string + want []string + }{ + {"where s = @status", []string{"@status"}}, + {"where s = sqlc.arg('status')", []string{"sqlc.arg('status')"}}, + {"where s = sqlc.arg(status)", []string{"sqlc.arg(status)"}}, + {"where s = sqlc.narg('note')", []string{"sqlc.narg('note')"}}, + {"where id in (sqlc.slice('ids'))", []string{"sqlc.slice('ids')"}}, + {"where a = @x and b = @y", []string{"@x", "@y"}}, + {"where n = sqlc.arg('c.name')", []string{"sqlc.arg('c.name')"}}, + // A cast follows the marker as ordinary text. + {"where n = sqlc.arg('x')::text", []string{"sqlc.arg('x')"}}, + // Operators and MySQL variables that begin with @ are not binds. + {"where tags @> '{a}'", nil}, + {"select @@version", nil}, + // A marker inside a quoted span is text. + {"select '@status', \"@a\", `@b`", nil}, + {"select /* @status */ 1", nil}, + // A schema-qualified call that is not one of the three forms stays opaque, which is + // what leaves sqlc.embed — a result-column construct, not a bind — alone. + {"select sqlc.args('x')", nil}, + {"select sqlc.embed(authors), id from authors", nil}, + } + for _, c := range cases { + t.Run(c.src, func(t *testing.T) { + got := named(scanAllNamed(t, c.src)) + if len(got) != len(c.want) { + t.Fatalf("markers = %q, want %q", got, c.want) + } + for i := range c.want { + if got[i] != c.want[i] { + t.Errorf("marker %d = %q, want %q", i, got[i], c.want[i]) + } + } + }) + } +} + +// Under the two-way syntax the same text is opaque, so a MySQL user variable — which a +// bind marker is indistinguishable from — is usable there. +func TestNamedBindOnlyUnderSqlcNamed(t *testing.T) { + for _, src := range []string{"where s = @status", "where s = sqlc.arg('status')"} { + for _, tok := range scanAll(t, src) { + if tok.kind == token.NamedBind { + t.Errorf("scanAll(%q) produced a NamedBind (%q) under the two-way syntax", src, tok.text) + } + } + } +} + +// A marker may be written across lines, and skipping it must not lose the line count. +func TestNamedBindKeepsLineNumbers(t *testing.T) { + l := lexer.NewWithRules("a\nsqlc.arg(\n'x'\n)\nb", + bindsyntax.RulesFor(bindsyntax.SqlcNamed, "postgresql")) + var lastWord ast4Loc + for { + k := l.Next() + if k == token.EOF || k == token.Illegal { + break + } + if k == token.Word && l.Token() == "b" { + lastWord = ast4Loc{l.Location().Line, l.Location().Column} + } + } + if lastWord.line != 5 || lastWord.col != 1 { + t.Errorf("location of the word after the marker = %d:%d, want 5:1", lastWord.line, lastWord.col) + } +} + +type ast4Loc struct{ line, col int } diff --git a/internal/sqltmpl/parser/named_test.go b/internal/sqltmpl/parser/named_test.go new file mode 100644 index 0000000..2be2426 --- /dev/null +++ b/internal/sqltmpl/parser/named_test.go @@ -0,0 +1,128 @@ +package parser_test + +import ( + "strings" + "testing" + + "github.com/mpyw/bisql/bindsyntax" + "github.com/mpyw/bisql/internal/sqltmpl/ast" + "github.com/mpyw/bisql/internal/sqltmpl/parser" +) + +// The round-trip property has to hold under either syntax: a bind that carries its own name +// has no test literal, so BindValue.Text() is the marker alone. +func TestParseWithRules_roundTrips(t *testing.T) { + for _, src := range []string{ + "select id from users where status = @status", + "select id from users where id in (sqlc.slice('ids'))", + "select id from users where name = sqlc.arg('c.name')::text", + "select id from users\nwhere 1 = 1\n /*%if a*/ and s = @s /*%end*/\n /*%for k in ks*/ and n like @k /*%end*/", + "select id from users where tags @> '{a}' and @@version is not null", + } { + t.Run(src, func(t *testing.T) { + node, err := parser.ParseWithRules(src, bindsyntax.RulesFor(bindsyntax.SqlcNamed, "postgresql")) + if err != nil { + t.Fatalf("parse: %v", err) + } + if got := node.Text(); got != src { + t.Errorf("Text()\n got: %q\nwant: %q", got, src) + } + }) + } +} + +// Without a test literal there is nothing to read the shape from, so the slice form is what +// carries the request to expand. +func TestParseWithRules_expandList(t *testing.T) { + cases := map[string]bool{ + "where id in (sqlc.slice('ids'))": true, + "where id = @ids": false, + "where id = sqlc.arg('ids')": false, + "where id = sqlc.narg('ids')": false, + } + for src, want := range cases { + t.Run(src, func(t *testing.T) { + node, err := parser.ParseWithRules(src, bindsyntax.RulesFor(bindsyntax.SqlcNamed, "postgresql")) + if err != nil { + t.Fatalf("parse: %v", err) + } + binds := collectBinds(node) + if len(binds) != 1 { + t.Fatalf("found %d binds, want one", len(binds)) + } + if binds[0].ExpandList != want { + t.Errorf("ExpandList = %v, want %v", binds[0].ExpandList, want) + } + if binds[0].Test != nil { + t.Errorf("Test = %#v, want nil", binds[0].Test) + } + if binds[0].Expression != "ids" { + t.Errorf("Expression = %q, want ids", binds[0].Expression) + } + }) + } +} + +func TestParseWithRules_rejectsTestLiteralForms(t *testing.T) { + cases := []struct{ src, want string }{ + {"where s = /*status*/'active'", "the two-way bind directive"}, + {"limit /*^lim*/10", "literal interpolation"}, + } + for _, c := range cases { + t.Run(c.src, func(t *testing.T) { + _, err := parser.ParseWithRules(c.src, bindsyntax.RulesFor(bindsyntax.SqlcNamed, "postgresql")) + if err == nil { + t.Fatalf("want an error containing %q, got nil", c.want) + } + if !strings.Contains(err.Error(), c.want) { + t.Errorf("error = %q, want it to contain %q", err, c.want) + } + }) + } +} + +// collectBinds walks the tree and returns every bind it finds. +func collectBinds(n ast.Node) []ast.BindValue { + var out []ast.BindValue + var walk func(ast.Node) + walk = func(n ast.Node) { + switch v := n.(type) { + case ast.BindValue: + out = append(out, v) + case ast.Statement: + for _, c := range v.Nodes { + walk(c) + } + case ast.Paren: + walk(v.Node) + case ast.IfBlock: + walk(v.If) + for _, e := range v.Elseif { + walk(e) + } + if v.Else != nil { + walk(v.Else) + } + case ast.IfDirective: + for _, c := range v.Nodes { + walk(c) + } + case ast.ElseifDirective: + for _, c := range v.Nodes { + walk(c) + } + case ast.ElseDirective: + for _, c := range v.Nodes { + walk(c) + } + case ast.ForBlock: + walk(v.For) + case ast.ForDirective: + for _, c := range v.Nodes { + walk(c) + } + } + } + walk(n) + return out +} diff --git a/internal/sqltmpl/parser/parser.go b/internal/sqltmpl/parser/parser.go index 3e3e37f..40b8f51 100644 --- a/internal/sqltmpl/parser/parser.go +++ b/internal/sqltmpl/parser/parser.go @@ -10,16 +10,24 @@ import ( "regexp" "strings" + "github.com/mpyw/bisql/bindsyntax" "github.com/mpyw/bisql/internal/sqltmpl/ast" "github.com/mpyw/bisql/internal/sqltmpl/lexer" "github.com/mpyw/bisql/internal/sqltmpl/token" ) -// Parse turns a template string into the template tree. The result satisfies -// node.Text() == src, except that parser-level comments (/*%! ... */) and a trailing -// delimiter (;) are dropped. +// Parse turns a template string into the template tree using bisql's two-way bind syntax. +// The result satisfies node.Text() == src, except that parser-level comments (/*%! ... */) +// and a trailing delimiter (;) are dropped. func Parse(src string) (ast.Node, error) { - p := &parser{lex: lexer.New(src)} + return ParseWithRules(src, bindsyntax.Rules{Syntax: bindsyntax.TwoWay}) +} + +// ParseWithRules turns a template string into the template tree, reading the bind spellings +// the rules allow. Only the bind spelling differs: the block directives are comments under +// any syntax, so they parse identically. +func ParseWithRules(src string, rules bindsyntax.Rules) (ast.Node, error) { + p := &parser{lex: lexer.NewWithRules(src, rules), rules: rules} p.push(&statementReducer{}) node, err := p.parse() if err != nil { @@ -33,6 +41,7 @@ func Parse(src string) (ast.Node, error) { type parser struct { lex *lexer.Lexer + rules bindsyntax.Rules reducers []reducer stop token.Kind // why the parse loop ended: EOF, Delimiter, or CloseParen loc ast.Location @@ -74,7 +83,7 @@ func (p *parser) parse() (ast.Node, error) { p.stop = k return p.reduceAll() case token.OpenParen: - child := &parser{lex: p.lex} + child := &parser{lex: p.lex, rules: p.rules} child.push(&statementReducer{}) node, err := child.parse() if err != nil { @@ -98,6 +107,10 @@ func (p *parser) parse() (ast.Node, error) { if err := p.parseBind(); err != nil { return nil, err } + case token.NamedBind: + if err := p.parseNamedBind(); err != nil { + return nil, err + } case token.LiteralValue: if err := p.parseLiteral(); err != nil { return nil, err @@ -129,6 +142,10 @@ func (p *parser) parse() (ast.Node, error) { } func (p *parser) parseBind() error { + if p.rules.Syntax != bindsyntax.TwoWay { + return p.errf("the two-way bind directive %s is not available with the %s bind syntax; "+ + "write the bind as sqlc.arg(name)", p.tok, p.rules.Syntax) + } expr := strip(p.tok, "/*", "*/") if expr == "" { return p.errf("expression is not found in the bind value directive") @@ -137,7 +154,32 @@ func (p *parser) parseBind() error { return nil } +// parseNamedBind handles a bind that carries its own name. There is no test literal to +// collect, so the node is complete on the spot and needs no reducer; a trailing cast is +// ordinary opaque text that follows it. +func (p *parser) parseNamedBind() error { + m, ok := p.rules.Recognize(p.tok) + if !ok { + return p.errf("malformed bind %q", p.tok) + } + p.pushNode(ast.BindValue{ + Loc: p.loc, + Token: p.tok, + Expression: m.Name, + ExpandList: m.Kind == bindsyntax.Slice, + }) + return nil +} + func (p *parser) parseLiteral() error { + if p.rules.Syntax != bindsyntax.TwoWay { + // The value is inlined as text rather than bound, so a static analyzer reading the + // template sees a constant and can check nothing about it. Refusing keeps the + // guarantee that every value in the query is one such a tool has vouched for. + return p.errf("literal interpolation %s is not available with the %s bind syntax; "+ + "bind the value as a parameter, or use a whitelisted /*%%if*/ toggle for an "+ + "identifier or a sort direction", p.tok, p.rules.Syntax) + } expr := strip(p.tok, "/*^", "*/") if expr == "" { return p.errf("expression is not found in the literal value directive") diff --git a/internal/sqltmpl/render/render.go b/internal/sqltmpl/render/render.go index a3b5c42..d72c8fd 100644 --- a/internal/sqltmpl/render/render.go +++ b/internal/sqltmpl/render/render.go @@ -123,43 +123,28 @@ func (r *renderer) eval(exprStr string) (any, error) { return v, nil } -// visitBind expands into an IN list when the test literal is a parenthesized group, and -// otherwise binds the value as a single parameter (so a slice becomes one array parameter, -// e.g. Postgres `= ANY($1::type[])`). Whether the target dialect supports array binding is -// the driver's concern. +// visitBind expands the value into a placeholder list when the bind asks for it, and +// otherwise binds it as a single parameter (so a slice becomes one array parameter, e.g. +// Postgres `= ANY($1::type[])`). Whether the target dialect supports array binding is the +// driver's concern. +// +// Who writes the surrounding parentheses differs by syntax, and it has to: a parenthesized +// test literal *is* the parentheses, so the renderer replaces them, while a bind spelled in +// the SQL has to be valid SQL before rendering — `in (sqlc.slice('ids'))` needs its parens +// written — so emitting another pair would double them. func (r *renderer) visitBind(node ast.BindValue) error { v, err := r.eval(node.Expression) if err != nil { return err } - if _, isParen := node.Test.(ast.Paren); isParen { - elems, ok := asIterable(v) - if !ok { - elems = []any{v} // scalar with a paren test -> single-element list - } + switch _, isParen := node.Test.(ast.Paren); { + case isParen: r.emit("(") - if len(elems) == 0 { - r.emit("null") - } - for i, e := range elems { - if i > 0 { - r.emit(", ") - } - if tup, ok := asIterable(e); ok { // a multi-column row, e.g. (a,b) IN ((1,2),(3,4)) - r.emit("(") - for j, te := range tup { - if j > 0 { - r.emit(", ") - } - r.bindOne(te) - } - r.emit(")") - } else { - r.bindOne(e) - } - } + r.bindList(v) r.emit(")") - } else { + case node.ExpandList: + r.bindList(v) + default: r.bindOne(v) } for _, c := range node.Trailing { @@ -170,6 +155,37 @@ func (r *renderer) visitBind(node ast.BindValue) error { return nil } +// bindList emits v as a comma-separated run of placeholders, without parentheses of its +// own. An empty list has no placeholders to emit and would leave the enclosing `in ()` +// invalid, so it renders as null — which matches nothing, the same as an empty IN list +// means. +func (r *renderer) bindList(v any) { + elems, ok := asIterable(v) + if !ok { + elems = []any{v} // a scalar where a list was asked for -> a one-element list + } + if len(elems) == 0 { + r.emit("null") + } + for i, e := range elems { + if i > 0 { + r.emit(", ") + } + if tup, ok := asIterable(e); ok { // a multi-column row, e.g. (a,b) IN ((1,2),(3,4)) + r.emit("(") + for j, te := range tup { + if j > 0 { + r.emit(", ") + } + r.bindOne(te) + } + r.emit(")") + } else { + r.bindOne(e) + } + } +} + func (r *renderer) visitLiteral(node ast.LiteralValue) error { v, err := r.eval(node.Expression) if err != nil { diff --git a/internal/sqltmpl/token/token.go b/internal/sqltmpl/token/token.go index 5a62eab..e6cf9ae 100644 --- a/internal/sqltmpl/token/token.go +++ b/internal/sqltmpl/token/token.go @@ -2,9 +2,10 @@ // // bisql does not parse SQL as a grammar and recognizes no clause keywords or connectors: // it removes nothing implicitly (the explicit-model design). The lexer -// only distinguishes directive comments, plain comments, string literals, and parentheses -// (needed to delimit a bind directive's test value and to detect IN-list expansion); -// everything else passes through as Word / Other / Space / Eol. +// only distinguishes directive comments, plain comments, string literals, parentheses +// (needed to delimit a bind directive's test value and to detect IN-list expansion), and — +// under a bind syntax that spells binds in the SQL rather than in a comment — the bind +// markers themselves; everything else passes through as Word / Other / Space / Eol. package token // Kind is a token kind. @@ -27,7 +28,11 @@ const ( MultiLineComment // /* ... */ (a plain block comment, not a directive) // directives - BindValue // /* expr */literal + BindValue // /* expr */literal + // NamedBind is a bind that carries its own name and needs no test literal: + // @name, sqlc.arg('name'), sqlc.narg('name'), sqlc.slice('name'). It is opaque + // text rather than a comment, and is only recognized under bindsyntax.SqlcNamed. + NamedBind LiteralValue // /*^ expr */literal ParserComment // /*%! ... */ (also hosts the @include preprocessor directive) If // /*%if e*/