fix(mysql_cdc): accept unicode table names - #4745
Open
samarth70 wants to merge 1 commit into
Open
Conversation
MySQL permits two sets of characters in unquoted identifiers: the ASCII set [0-9a-zA-Z$_] and the extended range U+0080 to U+FFFF. validateTableName only implemented the ASCII half, so a table named cafe with an accent, or one named in Japanese or Cyrillic, was rejected before the stream started and could not be captured at all. Replace the two regexes with explicit rune checks that also accept the extended range, and reject supplementary characters (U+10000 and above) and malformed UTF-8, neither of which MySQL permits.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
mysql_cdcrejects table names that MySQL itself accepts, so those tables cannot be captured at all.MySQL permits two sets of characters in unquoted identifiers (Schema Object Names):
validateTableNameimplements only the first line:Anything in the extended range fails one of the two checks. Every one of these is a legal MySQL table name:
caféinvalid table name日本語テーブルinvalid start char in mysql table nameПользователиinvalid start char in mysql table nameorders_ñ_2024invalid table nameThe check runs in
input_mysql_stream.gowhile the input is being built, so this is not a per-row warning: the connector refuses to start and the table can never be streamed.Fix
Replace the two regexes with explicit rune checks covering both sets MySQL documents. Supplementary characters (U+10000 and above) stay rejected, since MySQL does not permit them in identifiers either, and malformed UTF-8 is now rejected up front, because ranging over a string yields
utf8.RuneError, which sits inside the extended range and would otherwise be accepted.Everything the existing tests assert is unchanged: empty names, names over 64 characters,
@, spaces, and hyphens are all still rejected.I deliberately did not change the leading-digit rule. MySQL does allow identifiers to begin with a digit ("unless quoted may not consist solely of digits"), so
2usersis arguably valid too, butTestValidateTableNameexplicitly asserts it is rejected. That looks like an intentional choice rather than an oversight, so I left it alone. Happy to relax it in this PR if you would rather the validator matched the spec on that point as well.Testing
Four cases added for the extended range (accented Latin, Japanese, Cyrillic, and a mixed ASCII/extended name), plus three for the boundaries: length counted in runes, supplementary characters rejected, and malformed UTF-8 rejected.
The four extended-range cases fail on current
main:and pass with this change.
go test ./internal/impl/mysql/is green,gofmtandgo vetare clean.