Skip to content

[improve][io] Migrate Debezium from the 3.4.x line to 3.6.0.Final #63

Description

@david-streamlio

Motivation

The repository pins debezium = "3.4.2.Final" (released 2026-03-02). The current Debezium release is 3.6.0.Final (2026-07-01), so we are two minor versions behind.

Debezium has no LTS line — fixes land on the newest minor only. Once #64 bumps us to 3.4.3.Final (2026-03-30, the last patch the 3.4 line will ever receive), the 3.4 branch is end-of-line and we stop receiving bug and security fixes entirely. Each additional minor release makes the eventual migration more expensive.

Scope

Bump debezium in gradle/libs.versions.toml from 3.4.3.Final to 3.6.0.Final and fix any resulting breakage.

The connector set is unchanged. Comparing the debezium-bom contents for 3.4.2 and 3.6.0, the only addition is debezium-connector-common (an internal refactor). No wrapper module gains or loses a target.

Risks

The real risk is our coupling to Debezium internals, which are not API-stable across minor versions:

  • debezium/core implements the schema-history SPI (PulsarSchemaHistory) against io.debezium.storage / SchemaHistory interfaces.
  • kafka-connect-adaptor (AbstractKafkaConnectSource) starts Debezium tasks directly and depends on connector/task config-key conventions.
  • DebeziumSource subclasses assert on Debezium's connector.class / task.class values.

Issue #61 is a concrete example of this coupling already breaking against 3.4.x (multi-partition SQL Server connectors never receive task.id because we bypass taskConfigs()). Expect similar sharp edges on a minor bump; #61 should ideally be fixed first, or at least understood, since it touches the same task-startup path.

Prerequisites

The integration tests added in #58 (MongoDB), #60 (SQL Server), and #62 (Oracle) should be merged before this migration. Prior to those, a Debezium bump was validated only by the MySQL and Postgres tests. With all five source connectors exercised against real databases via Testcontainers, this upgrade either passes CI across the board or points directly at the connector that broke.

Note: this will not fix the Testcontainers conflict

debezium-bom:3.4.2.Final pins Testcontainers core to 2.0.2; 3.6.0.Final (via debezium-build-parent) pins 2.0.3. Both are 2.x, while the specialized Testcontainers modules (e.g. org.testcontainers:oracle-free) still target 1.21.x and reference shaded classes removed in 2.x. Debezium-module integration tests must keep using the GenericContainer pattern (as debezium/mysql and debezium/oracle do) regardless of this upgrade — see the discussion in #62.

Suggested order

  1. Merge [improve][test] Add Debezium MongoDB source integration test #58, [improve][test] Add Debezium SQL Server source integration test #60, [improve][test] Add Debezium Oracle source integration test #62 (integration-test safety net)
  2. Land the 3.4.3.Final patch bump (low risk, free fixes)
  3. Address or characterize [Bug] Debezium SQL Server source fails to start: multi-partition connector never receives task.id #61
  4. Migrate to 3.6.0.Final and let the five connector integration tests validate it

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions