fix: adjust scanner for correct parsing of dots, and adjust grammar for consistent highlighting of commands #20
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…or consistent highlighting, update tests
…t inline. Before, these nodes were grouped under system_defined_function and marked as inline, but for Language server purposes, it is helpful to have access to these nodes.
Having token.immediate() as the first thing for a node can lead to undefined behavior, in this case it allowed method_name in a class method call to be nothing.
…if the next node is an expression. Before, relative dot methods and relative dot procedures were incorrectly only working in set commands if there was no whitespace between the = and the expression. That is because the scanner was incorrectly consuming any dots as potential dotted statements. I fixed this by making sure the dots aren't consumed if the next node is an expression (as the only possible dotted statement is a statement, not an expression).
hkimura-intersys
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Overview
The main changes in this PR include:
Scanner Fix
Before, relative dot methods and relative dot procedures were incorrectly only working in set commands if there was no whitespace between the = and the expression, like this:
If there was any whitespace at all, the dots were consumed by the scanner, leading to errors or more subtle incorrect results:
In this example, the rhs should be a relative dot property, but since the dots are consumed, the grammar only sees the three and returns that it is a numeric literal:
In this example, we get an error because since the dots were consumed by the scanner, the grammar doesn't match anything:
I fixed this by adjusting the scanner to NOT consume any dots if the next node is an expression. This works because if the next node is an expression, we know whatever we are parsing cannot be a dotted statement. This change fixes our issues, and now the relative dot methods and relative dot parameters are being parsed as expected:
Consistent Highlighting
Before, some commands didn't have the command_name field, leading to inconsistent highlighting. (see picture below):
In this example, set is highlighted a specific color, and write is not.
With this fix, they are highlighted the same color:
