Skip to content

[CDC reliability] Fail fast on invalid Kafka replica retry settings #250

Description

@seonghobae

Buyer / reliability gap

On protected develop@942d164ceb7af497bb28a95f842db96429d1bdc4, KafkaConfig.kafkaListenerErrorHandler(...) passes deployment-owned xtrmetl.replica.kafka.retry-backoff-ms and xtrmetl.replica.kafka.retry-max-attempts directly into Spring Framework FixedBackOff without validating that either value is non-negative.

Spring Framework 6.2.18 does not validate these constructor arguments: it stores them directly. Its FixedBackOffExecution.nextBackOff() increments the attempt counter and returns the configured interval only while currentAttempts <= maxAttempts; otherwise it returns STOP. Therefore a negative max-attempts value silently disables the configured retry path, while a negative interval can survive construction and only fail or behave invalidly when an actual retry is attempted. This turns a deployment typo into runtime retry/DLT behavior rather than a deterministic startup/configuration failure.

The existing metadata describes these settings as a backoff interval and maximum retry attempts, and current tests cover the ordinary positive values but not invalid values.

RCA

  • Immediate cause: the two raw long values are forwarded directly to new FixedBackOff(retryBackoffMs, retryMaxAttempts).
  • Technical root cause: product configuration validity is delegated to a framework type whose 6.2.18 constructor deliberately performs no argument validation.
  • Control failure: KafkaConfigTest verifies the resulting FixedBackOff for 1000/30 only; no fail-closed invalid-configuration contract exists.
  • Impact: an operator can unknowingly disable retries with a negative retry count or defer an invalid negative delay to the live error path, undermining predictable poison-message recovery and DLT behavior.

Bounded remediation

Add production-boundary tests first, then reject negative retry backoff and negative retry attempts before constructing the recoverer/error handler. Keep zero valid: zero backoff means immediate retries, and zero max attempts means deliberate no-retry/DLT behavior. Preserve all existing positive/default values, non-retryable exception classification, DLT routing, listener concurrency, and acknowledgment semantics.

Use stable configuration-key-bearing IllegalArgumentException messages without echoing any unrelated configuration or payload data. Add beginner-readable Javadocs to the touched public bean factory method. Do not invent an arbitrary upper bound or reinterpret Spring's explicit unlimited sentinel in this slice.

Acceptance

  • RED reaches the real KafkaConfig.kafkaListenerErrorHandler(...) production boundary with negative backoff and negative max-attempts values and proves protected code currently accepts them;
  • GREEN rejects each invalid value before FixedBackOff construction with a stable message naming the responsible configuration key;
  • zero and existing positive values remain accepted with unchanged semantics;
  • focused CDC tests and full reactor pass on the exact candidate;
  • current dependency/SBOM/SAST/security evidence is regenerated; synthetic-merge-only evidence, incomplete dependency materialization, vacuous coverage, stale evidence, or absent independent approval is not promoted to protected-merge authority.

Primary upstream evidence: Spring Framework 6.2.18 FixedBackOff source and API documentation.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions