Skip to content

fix(mysql_cdc): accept unicode table names - #4745

Open
samarth70 wants to merge 1 commit into
redpanda-data:mainfrom
samarth70:fix/mysql-cdc-unicode-table-names
Open

fix(mysql_cdc): accept unicode table names#4745
samarth70 wants to merge 1 commit into
redpanda-data:mainfrom
samarth70:fix/mysql-cdc-unicode-table-names

Conversation

@samarth70

Copy link
Copy Markdown

Problem

mysql_cdc rejects table names that MySQL itself accepts, so those tables cannot be captured at all.

MySQL permits two sets of characters in unquoted identifiers (Schema Object Names):

  • ASCII: [0-9,a-z,A-Z$_] (basic Latin letters, digits 0-9, dollar, underscore)
  • Extended: U+0080 .. U+FFFF

validateTableName implements only the first line:

if matched, _ := regexp.MatchString(`^[a-zA-Z_]`, tableName); !matched {
    return errInvalidTableStartChar
}
if matched, _ := regexp.MatchString(`^[a-zA-Z0-9_$]+$`, tableName); !matched {
    return errInvalidTableName
}

Anything in the extended range fails one of the two checks. Every one of these is a legal MySQL table name:

table result before
café invalid table name
日本語テーブル invalid start char in mysql table name
Пользователи invalid start char in mysql table name
orders_ñ_2024 invalid table name

The check runs in input_mysql_stream.go while the input is being built, so this is not a per-row warning: the connector refuses to start and the table can never be streamed.

Fix

Replace the two regexes with explicit rune checks covering both sets MySQL documents. Supplementary characters (U+10000 and above) stay rejected, since MySQL does not permit them in identifiers either, and malformed UTF-8 is now rejected up front, because ranging over a string yields utf8.RuneError, which sits inside the extended range and would otherwise be accepted.

Everything the existing tests assert is unchanged: empty names, names over 64 characters, @, spaces, and hyphens are all still rejected.

I deliberately did not change the leading-digit rule. MySQL does allow identifiers to begin with a digit ("unless quoted may not consist solely of digits"), so 2users is arguably valid too, but TestValidateTableName explicitly asserts it is rejected. That looks like an intentional choice rather than an oversight, so I left it alone. Happy to relax it in this PR if you would rather the validator matched the spec on that point as well.

Testing

Four cases added for the extended range (accented Latin, Japanese, Cyrillic, and a mixed ASCII/extended name), plus three for the boundaries: length counted in runes, supplementary characters rejected, and malformed UTF-8 rejected.

The four extended-range cases fail on current main:

--- FAIL: TestValidateTableName/Valid_table_name_with_accented_latin_characters
--- FAIL: TestValidateTableName/Valid_table_name_starting_with_an_extended_character
--- FAIL: TestValidateTableName/Valid_table_name_with_cyrillic_characters
--- FAIL: TestValidateTableName/Valid_table_name_mixing_ascii_and_extended_characters

and pass with this change. go test ./internal/impl/mysql/ is green, gofmt and go vet are clean.

MySQL permits two sets of characters in unquoted identifiers: the ASCII
set [0-9a-zA-Z$_] and the extended range U+0080 to U+FFFF.
validateTableName only implemented the ASCII half, so a table named
cafe with an accent, or one named in Japanese or Cyrillic, was rejected
before the stream started and could not be captured at all.

Replace the two regexes with explicit rune checks that also accept the
extended range, and reject supplementary characters (U+10000 and above)
and malformed UTF-8, neither of which MySQL permits.
@CLAassistant

CLAassistant commented Aug 31, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants