Skip to content

Keep tests from every declaration of a duplicated column - #16322

Open
CallumFriend-MP wants to merge 3 commits into
dbt-labs:mainfrom
CallumFriend-MP:callumfriend/duplicate-column-tests
Open

CallumFriend-MP wants to merge 3 commits into
dbt-labs:mainfrom
CallumFriend-MP:callumfriend/duplicate-column-tests

Conversation

@CallumFriend-MP

@CallumFriend-MP CallumFriend-MP commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

Fixes #16313.

When the same column name appears twice in one columns: list, the v2 parser keeps only the tests from the last declaration and drops the earlier one's. Parse exits 0 with nothing logged, so a data test disappears from the manifest and stops running.

column_tests_inner collects column entries straight into a BTreeMap<String, ColumnTestEntry> keyed by column name, so the later entry evicts the earlier one wholesale. This collects onto a single entry instead. The column's own attributes (quote, tags, legacy_syntax_handling) stay last-wins, which is how its description already resolves under both parsers.

This isn't #15841. That one covered tests whose generated names collide across different models, and it's fixed. This reproduces on 2.0.0rc2 which contains it, and here the two names don't collide at all.

Scope

The versioned-columns path in persist_generic_data_tests.rs has the same insert-keyed-by-name shape. I left it alone on purpose: an existing entry there can come from version inheritance, where overriding is the point. Happy to extend it if you'd rather the two were consistent.

Verification

cargo check -p dbt-parser is clean. cargo test -p dbt-parser --lib gives 398 passed, 0 failed, 5 ignored, same as main.

Built a binary from this branch and ran it on the repro from #16313:

dbt-core 1.12.0  [not_null_orders_status_code.9c7afb188a, unique_orders_status_code.01ce9acdea]
stock 2.0.0rc2   [unique_orders_status_code.01ce9acdea]
this branch      [not_null_orders_status_code.9c7afb188a, unique_orders_status_code.01ce9acdea]

Surviving column description is second declaration in all three, so last-wins is preserved.

This is one of two options

#16313 sets out two ways to close this. This PR does the first:

  1. Match dbt-core, keep tests from every declaration. Lowest risk. No project changes behaviour except getting back a test it had lost.
  2. Reject the duplicate, so both engines raise instead of silently merging (dbt-core) or silently dropping (v2).

I think 2 is the better end state. A column declared twice with two different descriptions is nearly always a mistake, and right now neither engine says anything about it. But it breaks projects that parse clean today, including the one where I found this. So it would want a deprecation warning first and an error later, and it needs to land in both engines together. Otherwise it just swaps one v1/v2 divergence for another.

That's a maintainer call, not something to decide in a PR, so I went with the conservative one here. Happy to swap this for the deprecation-warning version, or stack it on top.

Pre-existing failure

cargo test -p dbt-parser --doc fails on resolve::validate_metrics::validate_metric_name (line 43) with E0432: unresolved import dbt_parser::resolve::validate_metric_name. It fails the same way on pristine main, so it isn't from this PR.

`column_tests_inner` collects column entries straight into a BTreeMap keyed
by column name, so a column declared twice in one `columns:` list lost the
earlier declaration's data tests. Parsing exits 0 with no warning, so the
test disappears from the manifest and never runs again.

dbt-core keeps the tests from both declarations. Collect onto a single
entry rather than overwriting; the column's own attributes stay last-wins,
matching how its description already resolves.

The versioned-columns path in persist_generic_data_tests.rs has the same
shape but is left alone: an existing entry there can come from version
inheritance, where overriding is the intended behaviour.

Fixes dbt-labs#16313.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[v2 Bug] Duplicate column declaration in one model silently drops the earlier declaration's data tests

1 participant