Better oracle statement splitting - #818
Open
icedtoast wants to merge 6 commits into
Open
Conversation
Help make the SQL Server Batch Splitting more robust by adding test cases which don't work well with the current regex-based replacement. These are taken from chucknorris/roundhouse#82
Contributor
Author
icedtoast
marked this pull request as draft
August 18, 2026 21:02
wokket
added
help wanted
Collaborator
|
I haven't touched an Oracle system in over 20 years... any experts out there please chime in! |
icedtoast
force-pushed
the
better-oracle-statement-splitting
branch
from
August 19, 2026 01:46
3c1c134 to
cf41848
Compare
Contributor
Author
|
Okay I think I have sorted out the specifics of Oracle behavior, notably it will split on both ; and / now. Probably worth a run on someones actual environment as a test. |
icedtoast
marked this pull request as ready for review
August 19, 2026 01:47
icedtoast
force-pushed
the
better-oracle-statement-splitting
branch
2 times, most recently
from
August 24, 2026 22:18
13f88ef to
e336c72
Compare
Hide the implementation details that it is implemented by regular expressions to allow easier implementation of different techniques in the future. Also a note on the terminology, it is probably better named batch splitting, but I wanted to keep the implementation as close to how it was originally; hence the same naming. Furthermore, dependent code doesn't care *how* the statements are split, just that they are - so this further encapsulates that decision and is a better abstraction point. Notably some tests have changed from using raw string literals, instead being parameterized to accept the line endings in different environments (i.e. Windows and everything else). This helps the tests be more consistent. (Thanks to @wokket for pointing this issue out: grate-devs#816 (comment) )
Although the Splitter name was inherited from the BatchSplitterReplacer tests, we want to more to more consistent naming used in the rest of grate. Thanks @wokket for the comment: grate-devs#816 (comment)
This handles nested comments and strings more reliably that a solely regex-based approach. It is also clearer. Regular expressions used to tokenize the sql statement, which is then parsed to ensure that it is only split when outside comments and string literals. A small behavior change is that tabs will longer be a batch on their own , i.e. whitespace only batches are dropped.
Add unit tests to cover edges exposed by SQL*Plus handling of the Oracle specific batch separators. Notably the current regex-based statement splitter fails these tests.
Notably this makes the batch splitting compatible with Oracle SQL*Plus. Important changes to be aware of is that it will *never* split with a / unless it is on its own line and not in a string. You can see the documentation on this here: https://docs.oracle.com/en/database/oracle/oracle-database/26/sqpug/slash.html https://docs.oracle.com/database/121/SQPUG/ch_four.htm#SQPUG395 This supercedes grate-devs#329 and helps with grate-devs#642 Building on those issues and changes it will also split batches on ; however it will retain the semicolon in the batch so that it won't cause a parser error with Oracle. Additionally this requires handling BEGIN, END and DECLARE blocks as the semicolons should be ignored in there. This does change behavior in that whitespace may be trimmed on batches, depending on where the ; or / separator was.
icedtoast
force-pushed
the
better-oracle-statement-splitting
branch
from
August 24, 2026 23:07
e336c72 to
0d5eb8d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is the changes for the Oracle statement splitter to be a tokenizer & parser. This builds on top of #816
I believe this will also supercede #329 and fix #642
However I haven't used Oracle for a long time, so I would appreciate someone checking if it works for them.
Thanks in advance for reviewing.