Refactor built-in functions from special lexer tokens to named functions#285
Closed
Copilot wants to merge 1 commit into
Closed
Refactor built-in functions from special lexer tokens to named functions#285Copilot wants to merge 1 commit into
Copilot wants to merge 1 commit into
Conversation
- Add BuiltinFunc enum and registry in internal/ast/builtins.go - Unify CallExpr and UserCallExpr into a single CallExpr with Builtin field - Remove F_* token constants and FIRST_FUNC/LAST_FUNC from lexer - Remove built-in function names from lexer keyword map (now plain NAME tokens) - Update parser to look up built-in functions by name from registry - Update resolver to switch on BuiltinFunc enum instead of lexer.Token - Update compiler to switch on BuiltinFunc enum instead of lexer.Token - Update tests to match new behavior Agent-Logs-Url: https://github.com/benhoyt/goawk/sessions/22e39b39-5167-4e71-934e-f869bc7a9467 Co-authored-by: benhoyt <999033+benhoyt@users.noreply.github.com>
Copilot created this pull request from a session on behalf of
benhoyt
May 14, 2026 20:21
View session
Owner
|
This was an interesting exercise, but if/when I do this I'll want to do it a somewhat different way to maintain backwards compatibility. I'll probably leave the existing builtins as keywords, but add new ones as a new type (or augment UserCallExpr). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Built-in functions (
substr,length,split, etc.) were special-cased as lexer tokens (F_SUBSTR,F_LENGTH,F_SPLIT, …) with per-function parsing logic and two separate AST node types (CallExprvsUserCallExpr). This refactors them to be resolved by name in the parser, unifying the AST representation.Changes
BuiltinFuncenum (internal/ast/builtins.go): 22 built-in values +BuiltinNonefor user-defined calls, with bidirectional name↔enum lookup viaBuiltinFuncByName()CallExpr: Replaces bothCallExpr{Func lexer.Token}andUserCallExpr{Name string}with a singleCallExpr{Builtin BuiltinFunc, Name string, Args []Expr, Pos Position}.Builtin == BuiltinNoneindicates a user-defined call.F_*token constants,FIRST_FUNC/LAST_FUNCsentinels, and built-in entries fromkeywordTokens. These names now lex as plainNAMEtokens.primary()dispatches tobuiltinCall()viaBuiltinFuncByName()lookup onNAMEtokens, preserving all per-function validation (lvalue constraints, array args, optional parens forlength, variadicsprintf, etc.).concat()no longer needs theFIRST_FUNC/LAST_FUNCrange check.ast.BuiltinFuncenum instead oflexer.Token.UserCallExprvisitor paths merged into the unifiedCallExprhandler.Before/After
Adding a new built-in function now requires a registry entry in
builtins.go+ a parse case inbuiltinCall(), rather than touching lexer, parser, compiler, and resolver token maps.