Skip to content

[Product/API] Separate live CDC target capability discovery from unwired canonical-write SPI stubs #202

Description

@seonghobae

Buyer / integrator truthfulness gap

Protected develop@622e5e6c3d534f230c390f10e3832efadfc01825 exposes GET /api/cdc/targets and registeredTargets in status from CdcTargetRegistry. The registry creates two objects:

  • KafkaCdcTargetConnectorscaffoldOnly() == false, but write(List<CanonicalChangeRecord>) always throws UnsupportedOperationException because live Kafka publication still occurs separately in CdcService using raw Debezium envelopes;
  • JdbcReplicaCdcTargetConnectorscaffoldOnly() == false, but write(...) always throws UnsupportedOperationException because the actual optional PostgreSQL replica path is separately owned by ProcessedDataReplicaApplier.

The current test explicitly asserts the contradiction for Kafka: non-scaffold status plus an always-unsupported write() method. The product paths themselves are real (PostgreSQL→Kafka raw publication and optional Kafka→PostgreSQL replica apply), but the public Java target connector SPI is not executable. An API consumer seeing only scaffoldOnly=false cannot distinguish a live product path from an unwired canonical-write adapter.

Status: known_gap. Do not “fix” this by marking supported product paths unsupported or by wiring a new canonical payload format without compatibility evidence.

RCA

  • Immediate cause: discovery objects are being used both as descriptors of an existing product target path and as implementations of an executable canonical-write SPI.
  • Technical root cause: capability discovery and execution authority are conflated in CdcTargetConnector (id/display/scaffoldOnly + write).
  • Control failure: scaffoldOnly is one boolean and cannot express “live through legacy/raw path, canonical SPI not wired”; current tests normalize this ambiguity instead of failing on it.
  • Commercial risk: Java integrators can call a type presented as non-scaffold and receive an unconditional runtime failure; HTTP target discovery cannot truthfully advertise the executable surface of the registered object.

Distinct remediation classes

Preferred — separate descriptor capability from executable adapter

Refactor so discovery can truthfully describe at least:

  • product path support (live, reference/scaffold, planned);
  • payload/contract mode (for example raw Debezium versus canonical record);
  • whether the registered Java adapter itself supports canonical batch write;
  • whether execution is currently owned by another explicit pipeline component.

This can be a descriptor type plus a separately injectable executable SPI, or a richer capability record with no unsupported method on non-executable descriptors. Preserve the live raw Kafka/JDBC product paths until a separately reviewed cutover exists.

Alternative — productionize the canonical target adapters

Wire maintained implementations only if compatibility, delivery/transaction semantics, credentials, backpressure, retry/idempotency, schema evolution and realistic integration tests are established. For Kafka, do not silently replace raw Debezium publication with canonical JSON and break existing consumers. For JDBC replica, do not pretend a generic canonical write is equivalent to the current bounded processed_data path.

Reject

  • merely changing scaffoldOnly() to true and thereby calling the genuinely live raw product target unsupported;
  • leaving scaffoldOnly=false while write() is permanently guaranteed to throw;
  • catching UnsupportedOperationException at callers and treating it as capability negotiation;
  • wiring canonical Kafka output without a versioned consumer migration;
  • adding fake always-success adapters to make discovery look complete.

TDD / acceptance

Before production refactor, add a fail-first contract that distinguishes product-path support from Java-adapter execution. Final acceptance must prove:

  1. /api/cdc/targets and status accurately describe the executable and payload-mode capability of each target;
  2. no object advertised as executable through a given SPI has an unconditional unsupported implementation for that SPI;
  3. live raw Kafka publication remains backward compatible until an explicit cutover contract changes it;
  4. optional JDBC replica support remains truthful about its currently bounded table/path semantics;
  5. Java API compatibility is reviewed because CdcTargetConnector is a public production type;
  6. unsupported/reference targets are either non-production descriptors or explicitly non-executable in machine-readable capability metadata;
  7. unit/integration/compatibility tests cover supported and unsupported modes without fake provider success;
  8. canonical PRD/TRD/Architecture/UML/API/connector support matrix/Traceability agree with protected implementation status;
  9. exact-source CI/security and non-vacuous coverage/review gates pass before protected integration.

Relationship / sequencing

Issue #153 owns the broader rule that permanent scaffold-only connectors must not remain misleading production endpoints. This issue is narrower: the two currently real CDC product paths are described by unwired Java write adapters, so simple removal/scaffold labeling would also be misleading. Resolve the capability/execution model before claiming the generic target SPI is production-ready. Do not race active connector-retirement PRs or the DLT slices #192/#197.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: apiAPI, protocol, event, or external contractarea: authAuthentication, authorization, identity, or tenant isolationarea: ci-cdCI, GitHub Actions, checks, release, or supply chainarea: securitySecurity boundary, hardening, or vulnerability preventionmaintenancepriority: mediumNormal-priority or P2 workstatus: triagedOpen issue has an organization taxonomy assignmenttype: featureNew or expanded product capability

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions