Skip to content

Fix charset-aware MySQL expressions and result encoding - #1854

Merged
ketor merged 4 commits into
dingodb:developfrom
ketor:fix/mysql-charset-followup
Sep 23, 2026
Merged

ketor merged 4 commits into
dingodb:developfrom
ketor:fix/mysql-charset-followup

Conversation

@ketor

@ketor ketor commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

Follow-up to #1853 (merged before the review fixes could be added).

  • Correct CHAR numeric rounding/decimal-text truncation and reject invalid byte values.
  • Return charset-aware VARCHAR from CONVERT(... USING ...) and CHAR(... USING ...), preserving strict Windows-1252 (latin1) and ASCII conversion rather than disguising text as binary.
  • Carry operand charset through CONCAT and use it for byte-sensitive LENGTH/HEX; expose expression charset in JDBC metadata. Promote mixed text without losing characters: Unicode wins over non-Unicode at equal coercibility, representable literals retain the expression charset, and ASCII can promote to latin1.
  • Encode text and prepared result rows in the negotiated character_set_results, advertise the matching wire collation, reject unrepresentable characters, and correct NOT_NULL and CONVERT_TZ nullable metadata.

Verification on the 172.30.14.10 QA executor after deploying the final candidate:

  • ./gradlew :dingo-calcite:test :dingo-driver:dingo-mysql-service:test :dingo-test:test :dingo-test:compileIntTestJava --console=plain — BUILD SUCCESSFUL. The new mixed-charset regression failed before the fix and passes afterward; the existing latin1+literal regression still passes.
  • Live MySQL CLI: latin1 € + utf8mb4 🙂 yields LENGTH=7, HEX=E282ACF09F9982; reversed yields F09F9982E282AC; three-operand concat yields LENGTH=9, HEX=E282ACF09F9982C2A3. Both ASCII/latin1 orders preserve the euro (8041 / 4180). Direct result rows decode as €🙂 and €A under utf8mb4 result collation 45.
  • Live Connector/J 8.0.33 server-prepared parameterized concat returns €🙂, HEX=E282ACF09F9982, Types.VARCHAR.
  • Earlier live checks: MySQL CLI UTF-8 results €, é; Connector/J direct and prepared VARCHAR text, nested LENGTH(CONCAT(...)) = 2, latin1 euro HEX = 80, numeric CHAR, invalid ASCII rejection, nullable invalid-zone CONVERT_TZ, and both SET NAMES latin1 / SET character_set_results=utf8mb4 result modes.

Additional fix bd854ca (mixed-charset CASE): SQL validation and Rex conversion now preserve Unicode regardless of branch order, including nullable ELSE branches; direct CASE projection metadata reports nullable correctly. QA verification after the change: ./gradlew :dingo-calcite:test --tests io.dingodb.calcite.TestDingoSqlValidator --tests io.dingodb.calcite.TestRex :dingo-test:test --tests io.dingodb.test.QuerySimpleExpressionTest --console=plain — BUILD SUCCESSFUL. Restarted QA executor live MySQL query returned F09F9982, LENGTH=4, reverse-case E282AC, and nullable-case F09F9982.

@ketor
ketor merged commit bb96a74 into dingodb:develop Sep 23, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant