Skip to content

[CDC data integrity] Reject non-integral or out-of-range replica row IDs instead of coercing them #237

Description

@seonghobae

Buyer / data-integrity defect

Protected develop@622e5e6c3d534f230c390f10e3832efadfc01825 parses CDC replica row identifiers in ProcessedDataReplicaApplier.extractLong(...) with:

if (value.isNumber()) {
    return value.longValue();
}

Jackson JsonNode.longValue() is a numeric coercion, not an exact-integral validation boundary. As written, a numeric Debezium key/value such as a fractional ID can be coerced to a different long, and values outside the signed 64-bit domain can likewise lose their original identity rather than fail closed. Because this value becomes the JDBC-bound primary-key selector for replica upsert/delete, coercion can target the wrong row.

Fresh test inventory on the same protected head covers ordinary integer IDs, malformed JSON, missing IDs, create/update/delete, topic allow-listing and table-name safety, but contains no fractional/out-of-range/exactness cases.

RCA

  • Immediate cause: the parser accepts every Jackson numeric node and delegates to longValue().
  • Technical root cause: representation parsing and primary-key domain validation are conflated; "is numeric" is treated as equivalent to "is exactly representable as a signed 64-bit integral identifier".
  • Control failure: current tests assert successful small integer IDs but do not test decimal, exponent, or overflow boundaries.
  • Impact: a malformed or schema-drifted CDC record can be applied to a different replica row. This is a data-integrity failure, not merely input normalization.

Bounded remedy

After the current ProcessedDataReplicaApplier writer lane clears, add fail-first tests at the real apply(...) boundary and accept only identifiers that are exactly integral and within the Java/PostgreSQL bigint domain. Textual IDs may continue to use strict Long.parseLong semantics if compatibility requires them. Numeric JSON must not truncate fractions, wrap/clip overflow, or silently reinterpret exponent values that are not exact in-range integers.

The safest implementation should use Jackson's exact integral/range predicates or exact BigInteger conversion followed by signed-64-bit bounds checking; do not parse through floating point. Invalid IDs must cause zero JDBC mutation and retain the existing confidentiality rule (no raw ID/event/parser diagnostic in ordinary logs).

TDD acceptance

  1. RED proves a fractional key ID (for example 1.5) is not allowed to become row 1.
  2. RED proves numeric values above Long.MAX_VALUE and below Long.MIN_VALUE cannot wrap/clip into another row ID.
  3. Exact boundary values Long.MIN_VALUE and Long.MAX_VALUE remain valid if the database schema accepts bigint.
  4. Ordinary integral numeric and strict decimal-text IDs retain current behavior.
  5. When key ID is invalid, define explicitly whether fallback to an exact after.id is allowed; never let an invalid-but-present key silently target a coerced row. Test the chosen precedence.
  6. Delete and upsert paths both fail/skip safely with zero JDBC writes for invalid identity.
  7. No rejected ID, event payload, topic/customer identifier, parser message, or stack trace is added to ordinary logs.
  8. Focused/full CDC tests, exact non-vacuous owned-production coverage, dependency/SBOM/SAST/security, literal-source and independent-review gates regenerate before protected integration.

Writer / sequencing boundary

Current Draft PR #174 owns ProcessedDataReplicaApplier.java for diagnostic confidentiality, so source mutation is defer_until_trigger while that exact path is actively owned. Do not race or broaden #174. Re-read the integrated/current file and tests after #174 stabilizes/integrates; then implement this as a separate, narrow data-integrity slice unless #174's writer has already supplied equivalent exact-ID validation.

Relationships: #174 diagnostic confidentiality; #188 recovery/consistency; #159 canonical data/quality traceability.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: authAuthentication, authorization, identity, or tenant isolationarea: ci-cdCI, GitHub Actions, checks, release, or supply chainarea: dependenciesDependency or lockfile maintenancearea: securitySecurity boundary, hardening, or vulnerability preventionbugSomething isn't workingpriority: mediumNormal-priority or P2 workstatus: triagedOpen issue has an organization taxonomy assignmenttype: featureNew or expanded product capability

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions