Skip to content

Kafka gateway: Kafka Produce API (key 0) — bridge to Iggy send_messages #3535

Closed

Description

@ryerraguntla

Part of #3560.

Summary

Wire Kafka Produce (API key 0) to IggyBridge::produce() — decode RecordBatch payloads and send to Iggy with balanced partitioning.

Motivation

Core write path for Kafka migration: existing producers must work without code changes.

Depends on

Scope

  • Decode Produce v3–v9 (scoped range); preserve RecordBatch as opaque bytes where possible (hot path)
  • Map Kafka topic + partition → Iggy stream/topic/partition
  • Partitioning: balanced / keyed per Kafka record (document semantics)
  • Produce response with correct base offset / error per partition
  • KAFKA-18659: ApiVersions advertises Produce min=0 for librdkafka compatibility (encode layer)
  • Integration test: produce via gateway → verify message in Iggy (SDK or CLI)

Acceptance criteria

  • kafka-console-producer.sh or kcat -P can send to a topic through the gateway
  • Messages visible in Iggy stream/topic
  • At-least-once documented (Phase 1)

References

Activity

  1. added
    gatewaysproxy gateways for other protocols like kafka
    on Jun 23, 2026
  2. changed the title [-]feat(gateways): Kafka Produce API (key 0) — bridge to Iggy send_messages[/-] [+]Kafka gateway: Kafka Produce API (key 0) — bridge to Iggy send_messages[/+] on Jun 24, 2026
  3. krishvishal commented on Sep 21, 2026

    @krishvishal
    Member

    I'm picking this up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

gatewaysproxy gateways for other protocols like kafka

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions