Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 26 additions & 1 deletion Upgrades/Karafka/2.6.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,30 @@ config.pause.max_timeout = 30_000
config.pause.with_exponential_backoff = true
```

### Per-Topic Pause Configuration Nested, Flat Readers Removed

Unlike the global setting above, this change was **not** previously deprecated with a warning period - it will break silently if you rely on the old flat form.

The flat per-topic pause readers (`topic.pause_timeout`, `topic.pause_max_timeout`, `topic.pause_with_exponential_backoff`) are removed. Use the nested `topic.pause.*` namespace instead:

```ruby
# Before (no longer works)
topic.pause_timeout
topic.pause_max_timeout
topic.pause_with_exponential_backoff

# After
topic.pause.timeout
topic.pause.max_timeout
topic.pause.with_exponential_backoff
```

Additionally:

- `topic.to_h` now emits a nested `pause:` hash instead of the flat keys, so any code inspecting `topic.to_h[:pause_timeout]` and similar needs updating.
- `topic.pausing`/`topic.pausing?` are renamed to `topic.pause`/`topic.pause?`.
- The Pro Granular Backoffs override DSL is unchanged (`pause(timeout:, max_timeout:, with_exponential_backoff:)`). In OSS, `topic.pause` reflects the global `config.pause.*` settings; per-topic overriding remains a Pro feature.

### Minimum `karafka-rdkafka` Version

Karafka 2.6 requires `karafka-rdkafka` `>= 0.28.0`. This version exposes `Rdkafka::Consumer#list_offsets` and rebuilds consumer `#lag` on top of it, so lag reads issue a single batched end-offsets query instead of one watermark roundtrip per partition.
Expand Down Expand Up @@ -349,9 +373,10 @@ The swarm supervisor's `SHUTDOWN_GRACE_PERIOD` was increased from 1 second to 15

## Summary of Actions Required

For most applications, the upgrade from 2.5 to 2.6 requires two actions:
For most applications, the upgrade from 2.5 to 2.6 requires three actions:

- Update `config.pause_timeout`, `config.pause_max_timeout`, and `config.pause_with_exponential_backoff` to `config.pause.timeout`, `config.pause.max_timeout`, and `config.pause.with_exponential_backoff`.
- If you use the flat per-topic readers, update `topic.pause_timeout`, `topic.pause_max_timeout`, and `topic.pause_with_exponential_backoff` to `topic.pause.timeout`, `topic.pause.max_timeout`, and `topic.pause.with_exponential_backoff`; also update any code relying on `topic.pausing`/`topic.pausing?` (now `topic.pause`/`topic.pause?`) or on `topic.to_h`'s flat pause keys (now nested under `pause:`). This one was not previously deprecated, so it can break silently.
- Ensure your `karafka-rdkafka` dependency allows `>= 0.28.0`.

Additionally, review the Behavioral Changes section above for runtime differences (lag reporting, non-`StandardError` handling, and swarm shutdown timing) that do not require code changes but may affect monitoring or error handling.
Expand Down