Repository navigation
fix(sql): stop trusting identifier text in generated DDL - #598
Merged
Merged
Conversation
`ident` passed a name through untouched when it merely started and ended with
a quoting delimiter:
function isQuotedIdentifier(name) {
const first = name[0], last = name[name.length - 1];
return (first === '"' && last === '"')
|| (first === '`' && last === '`')
|| (first === '[' && last === ']');
}
Identifiers come from introspecting a source database, so a column is called
whatever somebody created — including something that only looks wrapped. A
column named
"a" INTEGER); DROP TABLE victim; --"
satisfied that check and was emitted verbatim, so the generated migration was
CREATE TABLE orders (
"a" INTEGER); DROP TABLE victim; --" INTEGER,
ok_col INTEGER
);
which is a valid CREATE, then a valid DROP, then a comment. Verified end to
end before the fix: a hostile source database, introspected normally, produced
a migration whose own splitter cut it into three statements and dropped a table
on the target. Dialect-independent — Postgres, SQLite, MySQL, Db2 and SQL
Server all emitted it.
The threat model is ordinary for this tool: compare against a source you do not
fully control (a vendor database, a restored dump, a shared or multi-tenant
one), then run the migration on your target.
The check now validates the wrapper instead of recognising its shape. Every
dialect here escapes the closing delimiter by doubling it (`""`, `` `` ``,
`]]`), so with the doubled pairs removed a surviving delimiter means the body
closes the identifier early and the rest is not part of the name. Such a name
is not well-formed and goes to the dialect's quoter to be escaped from scratch.
The pass-through that the old check existed for is kept, so ordinary output is
byte-identical: `"my col"`, `[my col]`, `` `my col` `` and the escaped forms
`"a""b"`, `[a]]b]`, `` `a``b` `` all still emit unchanged, a bare name stays
bare, and a name that genuinely needs quoting still gets it.
Escaping rather than rejecting also keeps the name faithful: the payload column
is now created on the target under its exact original name.
The new tests assert the structural property — the DDL stays one statement —
rather than the absence of the payload's text, which necessarily still appears
inside the escaped identifier. They fail 6/96 on the unfixed generator and pass
96/96 with it.
Not fixed here, and still open: the ClickHouse `createTableSuffix` builds its
PRIMARY KEY clause from `table.primaryKey.columns` directly, bypassing `ident`,
and the caller passes any ORDER BY entry containing a paren through unquoted so
that catalog expressions like `cityHash64(id)` survive. Both are injectable the
same way. Closing them means deciding how to tell a catalog-quoted identifier
or an expression from a raw name, which is a design call rather than a patch.
npx vitest run 6898 passed, 1 expected fail, 145 skipped (was 6883)
npx eslint . exit 0
npm run lint:security exit 0
npm run typecheck exit 0
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019ehcTb7vYMCawV1ajxwfxB
Contributor
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: 1260af63-4e24-4342-ad4e-7fcff85c459a) |
Two paths put raw catalog names into the ClickHouse `ENGINE … ORDER BY`
clause, bypassing `ident`. Both are injectable the same way as the quoted
identifier fixed in the previous commit.
PRIMARY KEY
pkCols = table.primaryKey?.columns ?? []
` PRIMARY KEY ${clickHouseOrderBySql(pkCols)}`
The dialect read the names off `table` itself, so they never met a quoter:
) ENGINE = MergeTree PRIMARY KEY id) ORDER BY (id) -- ORDER BY (id, ts);
`createTableSuffix` now receives `quotedPrimaryKey` from the caller, which is
the only place that holds `ident`, and is documented to use the arrays rather
than reading names off `table`. A primary key is a column list, so it is always
identifier-quoted, with no expression passthrough.
ORDER BY
The caller decided what was an expression by looking at the text:
orderSrc.map((c) => (/[()]/.test(c) ? c : this.ident(c)))
That exists because a sorting key may legitimately be `tuple()` or
`cityHash64(id)`, which quoting would break. But a column named
`id) ENGINE = MergeTree ORDER BY (id` also contains a paren, so it went through
raw and rewrote the clause.
Trust now comes from provenance, not from the text. `TableSchema.orderByFromCatalog`
marks entries that are ClickHouse's own rendering of `system.tables.sorting_key`
— already valid, correctly quoted SQL, and the only place an expression can come
from. The provider sets it there and deliberately leaves it unset for the
`system.columns.is_in_sorting_key` fallback, which pushes bare column names.
Marked entries pass through verbatim; everything else is quoted.
Call sites
Both callers were checked. The SQL editor's table blueprint
(`tableBlueprintSql.ts`) had the same paren rule and also handed the dialect raw
`primaryKey.columns`; its names are typed by the person building the table, never
a catalog expression, so they are now quoted with no passthrough. Its quoting
primitive (`quoteIdentifierIfNeeded`) was already sound — a strict plain-name
regex, otherwise always escaping — so no change was needed there.
Compatibility
Existing valid output is byte-identical, verified against the cases #594 added:
) ENGINE = MergeTree ORDER BY id;
) ENGINE = MergeTree ORDER BY (order_id, coupon_id);
) ENGINE = ReplacingMergeTree PRIMARY KEY id ORDER BY (id, code);
) ENGINE = MergeTree ORDER BY tuple();
and catalog expressions still survive: `cityHash64(id)`, `tuple()`,
`` `order id` ``. Bare, properly quoted and schema-qualified identifiers are
unaffected.
The new tests use payloads carrying a parenthesis, because the rule being
replaced passed any entry containing one through raw — without it they could not
tell a fixed generator from a broken one. 8 of them fail on the unfixed
generator and all pass with it; the compatibility and catalog-expression cases
pass either way by design.
Known trade-off: a catalog `primary_key` that is itself an expression is now
emitted quoted, so such a table needs manual review rather than silently
producing a key over a column that does not exist. PRIMARY KEY taking no
expressions is deliberate.
npx vitest run 6914 passed, 1 expected fail, 145 skipped (was 6898)
npx eslint . exit 0
npm run lint:security exit 0
npm run typecheck exit 0
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019ehcTb7vYMCawV1ajxwfxB
Contributor
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: 12cbc2d6-6065-42f4-a8d1-a65f93f8d8d0) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two commits closing the identifier-injection class in generated DDL. The first is generic; the second is the ClickHouse-specific remainder it originally left open.
1.
933bce4— a quoted identifier was trusted by its shapeidentpassed a name through untouched when it merely started and ended with a quoting delimiter:Identifiers come from introspecting a source database, so a column is called whatever somebody created — including something that only looks wrapped. A column named
"a" INTEGER); DROP TABLE victim; --"satisfied that check and was emitted verbatim:A valid
CREATE, then a validDROP, then a comment. Verified end to end: a hostile source database, introspected normally, produced a migration that Fox Schema's own splitter cut into three statements, and running it dropped a table on the target. Dialect-independent.The check now validates the wrapper. Every dialect escapes its closing delimiter by doubling it (
"",``,]]), so with the doubled pairs removed, a surviving delimiter means the body closes the identifier early and the rest isn't part of the name. Such a name goes to the dialect's quoter to be escaped from scratch:Escaping rather than rejecting keeps the name faithful — the payload column is still created on the target under its exact original name.
2.
d191571— ClickHouse's ENGINE clause bypassedidentPRIMARY KEY. The dialect read names off
tableitself, so they never met a quoter:createTableSuffixnow receivesquotedPrimaryKeyfrom the caller — the only place holdingident— and is documented to use the arrays rather than reading names offtable. A primary key is a column list, so it is always quoted, with no expression passthrough.ORDER BY. The caller decided what was an expression by looking at the text:
That exists because a sorting key may legitimately be
tuple()orcityHash64(id). But a column namedid) ENGINE = MergeTree ORDER BY (idalso contains a paren, so it went through raw and rewrote the clause.Trust now comes from provenance, not text.
TableSchema.orderByFromCatalogmarks entries that are ClickHouse's own rendering ofsystem.tables.sorting_key— already valid, correctly quoted SQL, and the only place an expression can come from. The provider sets it there and deliberately leaves it unset for thesystem.columns.is_in_sorting_keyfallback, which pushes bare column names. Marked entries pass through verbatim; everything else is quoted.Both call sites were checked
The SQL editor's table blueprint (
tableBlueprintSql.ts) had the same paren rule and handed the dialect rawprimaryKey.columns. Its names are typed by the person building the table, never a catalog expression, so they are now quoted with no passthrough. Its own quoting primitive (quoteIdentifierIfNeeded) was already sound — strict plain-name regex, otherwise always escaping — so nothing changed there.Compatibility
Existing valid output is byte-identical, checked against the cases #594 added:
Catalog expressions still survive (
cityHash64(id),tuple(),`order id`). Bare, properly quoted and schema-qualified identifiers are unaffected:"my col",[my col],`my col`,"a""b",[a]]b],`ab` `` all emit unchanged, and a bare name stays bare.Test plan
npx vitest run— 6914 passed, 1 expected fail, 145 skipped (was 6883 at branch point; +31 tests)npx eslint .— exit 0npm run lint:security— exit 0npm run typecheck— exit 0 (thecreateTableSuffixsignature changed)Tests assert the structural property — the DDL stays one statement — plus the exact quoted form, rather than the absence of the payload's text, which necessarily still appears inside the escaped identifier. Both sets were run against the unfixed code: 6 of 96 fail without commit 1, and 8 of the ClickHouse cases fail without commit 2. The compatibility and catalog-expression cases pass either way by design, guarding against over-quoting.
Known trade-off
A catalog
primary_keythat is itself an expression is now emitted quoted, so such a table needs manual review rather than silently producing a key over a column that doesn't exist. PRIMARY KEY taking no expressions is deliberate.Remaining injection risks — verified, not fixed here
Both are outside this change and need their own decision:
table.engineis interpolated raw into the suffix. A hostile engine string yields two statements:It comes from
system.tables.engine, which ClickHouse renders itself, so the engine name is from a fixed set — but engine parameters (ReplicatedMergeTree('/path', '{replica}'),Distributed(cluster, db, table)) are chosen at CREATE TABLE time, so this deserves a look. Validating it would need care not to break parameterised engines.A newline in a name escapes a
-- review:comment — and this one is dialect-independent and broader than ClickHouse.identquotes but does not escape newlines, and a--comment ends at the line break, so a table namedx\nDROP TABLE victim; --produces a migration containing a standalone, executable:Confirmed with the splitter. Every
-- review:/-- commentpath that interpolates a name is affected, which is many of them, so the fix belongs at the comment-emitting layer rather than here.🤖 Generated with Claude Code
https://claude.ai/code/session_019ehcTb7vYMCawV1ajxwfxB